[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786497534195564-24578-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 01:18:54 dist-test-slave-t3q3 krb5kdc[24588](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 01:18:54 dist-test-slave-t3q3 krb5kdc[24588](info): set up 2 sockets
Aug 12 01:18:54 dist-test-slave-t3q3 krb5kdc[24588](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260812 01:18:55.483559 24578 mini_kdc.cc:121] Time spent starting KDC: real 1.268s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786497534195564-24578-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786497534195564-24578-0/krb5kdc/test-user.keytab.
Aug 12 01:18:55 dist-test-slave-t3q3 krb5kdc[24588](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497535, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T01:18:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:18:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.0.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.0.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.0.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab.
Entry for principal kudu/127.24.0.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab.
I20260812 01:18:55.611274 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786497534195564-24578-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786497534195564-24578-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786497534195564-24578-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.190:41337
--webserver_interface=127.24.0.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab
--principal=kudu/127.24.0.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:41753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.0.190:41337
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:18:55.695457 24612 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:18:55.695720 24612 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:18:55.695761 24612 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:18:55.696944 24612 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:18:55.697016 24612 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:18:55.697041 24612 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:18:55.697062 24612 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:18:55.697091 24612 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:18:55.698572 24612 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:41753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786497534195564-24578-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786497534195564-24578-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.0.190:41337
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab
--principal=kudu/127.24.0.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.190:41337
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.0.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24612
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786497534195564-24578-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:18:55.698853 24612 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:18:55.699208 24612 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 01:18:55.705863 24612 server_base.cc:1034] running on GCE node
W20260812 01:18:55.706321 24627 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:18:55.706854 24623 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:18:55.706900 24624 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:18:55.707238 24612 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:18:55.707646 24612 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:18:55.710325 24612 hybrid_clock.cc:648] HybridClock initialized: now 1786497535710288 us; error 110 us; skew 500 ppm
Aug 12 01:18:55 dist-test-slave-t3q3 krb5kdc[24588](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497535, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:18:55.713850 24612 init.cc:377] Logged in from keytab as kudu/127.24.0.190@KRBTEST.COM (short username kudu)
I20260812 01:18:55.714471 24612 webserver.cc:460] Webserver started at http://127.24.0.190:40887/ using document root <none> and password file <none>
I20260812 01:18:55.714735 24612 fs_manager.cc:359] Metadata directory not provided
I20260812 01:18:55.714808 24612 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:18:55.714964 24612 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:18:55.716430 24612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786497534195564-24578-0/minicluster-data/master-0/data/instance:
uuid: "e139560df79e4bc9aca5aa50be7d853c"
format_stamp: "Formatted at 2026-08-12 01:18:55 on dist-test-slave-t3q3"
server_key: "5dcfa669797a8f6e52679b6ef80409af"
server_key_iv: "cbac81325de09ded08dd0b18a662f9ff"
server_key_version: "encryptionkey@0"
I20260812 01:18:55.717170 24612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786497534195564-24578-0/minicluster-data/master-0/wal/instance:
uuid: "e139560df79e4bc9aca5aa50be7d853c"
format_stamp: "Formatted at 2026-08-12 01:18:55 on dist-test-slave-t3q3"
server_key: "5dcfa669797a8f6e52679b6ef80409af"
server_key_iv: "cbac81325de09ded08dd0b18a662f9ff"
server_key_version: "encryptionkey@0"
I20260812 01:18:55.718847 24612 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 01:18:55.719657 24639 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:18:55.719973 24612 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:18:55.720120 24612 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786497534195564-24578-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786497534195564-24578-0/minicluster-data/master-0/wal
uuid: "e139560df79e4bc9aca5aa50be7d853c"
format_stamp: "Formatted at 2026-08-12 01:18:55 on dist-test-slave-t3q3"
server_key: "5dcfa669797a8f6e52679b6ef80409af"
server_key_iv: "cbac81325de09ded08dd0b18a662f9ff"
server_key_version: "encryptionkey@0"
I20260812 01:18:55.720208 24612 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786497534195564-24578-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786497534195564-24578-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786497534195564-24578-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:18:55.734697 24612 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:18:55.737211 24612 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:18:55.737375 24612 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:18:55.745518 24612 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.190:41337
I20260812 01:18:55.745559 24736 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.190:41337 every 8 connection(s)
I20260812 01:18:55.745954 24612 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
I20260812 01:18:55.746986 24737 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:18:55.747169 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 24612
I20260812 01:18:55.747278 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786497534195564-24578-0/minicluster-data/master-0/wal/instance
I20260812 01:18:55.748174 24578 external_mini_cluster.cc:1428] Setting key 77e58c435350a544784db144d22e2385
I20260812 01:18:55.750589 24737 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e139560df79e4bc9aca5aa50be7d853c: Bootstrap starting.
I20260812 01:18:55.751309 24737 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e139560df79e4bc9aca5aa50be7d853c: Neither blocks nor log segments found. Creating new log.
I20260812 01:18:55.751642 24737 log.cc:824] T 00000000000000000000000000000000 P e139560df79e4bc9aca5aa50be7d853c: Log is configured to *not* fsync() on all Append() calls
I20260812 01:18:55.752394 24737 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e139560df79e4bc9aca5aa50be7d853c: No bootstrap required, opened a new log
I20260812 01:18:55.754428 24737 raft_consensus.cc:348] T 00000000000000000000000000000000 P e139560df79e4bc9aca5aa50be7d853c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e139560df79e4bc9aca5aa50be7d853c" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 41337 } }
I20260812 01:18:55.754634 24737 raft_consensus.cc:374] T 00000000000000000000000000000000 P e139560df79e4bc9aca5aa50be7d853c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:18:55.754700 24737 raft_consensus.cc:729] T 00000000000000000000000000000000 P e139560df79e4bc9aca5aa50be7d853c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e139560df79e4bc9aca5aa50be7d853c, State: Initialized, Role: FOLLOWER
I20260812 01:18:55.754863 24737 consensus_queue.cc:260] T 00000000000000000000000000000000 P e139560df79e4bc9aca5aa50be7d853c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e139560df79e4bc9aca5aa50be7d853c" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 41337 } }
I20260812 01:18:55.754958 24737 raft_consensus.cc:388] T 00000000000000000000000000000000 P e139560df79e4bc9aca5aa50be7d853c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:18:55.755002 24737 raft_consensus.cc:482] T 00000000000000000000000000000000 P e139560df79e4bc9aca5aa50be7d853c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:18:55.755051 24737 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e139560df79e4bc9aca5aa50be7d853c [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:18:55.755585 24737 raft_consensus.cc:504] T 00000000000000000000000000000000 P e139560df79e4bc9aca5aa50be7d853c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e139560df79e4bc9aca5aa50be7d853c" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 41337 } }
I20260812 01:18:55.755708 24737 leader_election.cc:302] T 00000000000000000000000000000000 P e139560df79e4bc9aca5aa50be7d853c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e139560df79e4bc9aca5aa50be7d853c; no voters:
I20260812 01:18:55.755895 24737 leader_election.cc:288] T 00000000000000000000000000000000 P e139560df79e4bc9aca5aa50be7d853c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 01:18:55.755934 24746 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e139560df79e4bc9aca5aa50be7d853c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:18:55.756179 24746 raft_consensus.cc:686] T 00000000000000000000000000000000 P e139560df79e4bc9aca5aa50be7d853c [term 1 LEADER]: Becoming Leader. State: Replica: e139560df79e4bc9aca5aa50be7d853c, State: Running, Role: LEADER
I20260812 01:18:55.756212 24737 sys_catalog.cc:564] T 00000000000000000000000000000000 P e139560df79e4bc9aca5aa50be7d853c [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:18:55.756359 24746 consensus_queue.cc:237] T 00000000000000000000000000000000 P e139560df79e4bc9aca5aa50be7d853c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e139560df79e4bc9aca5aa50be7d853c" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 41337 } }
I20260812 01:18:55.756762 24746 sys_catalog.cc:455] T 00000000000000000000000000000000 P e139560df79e4bc9aca5aa50be7d853c [sys.catalog]: SysCatalogTable state changed. Reason: New leader e139560df79e4bc9aca5aa50be7d853c. Latest consensus state: current_term: 1 leader_uuid: "e139560df79e4bc9aca5aa50be7d853c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e139560df79e4bc9aca5aa50be7d853c" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 41337 } } }
I20260812 01:18:55.756862 24748 sys_catalog.cc:455] T 00000000000000000000000000000000 P e139560df79e4bc9aca5aa50be7d853c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e139560df79e4bc9aca5aa50be7d853c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e139560df79e4bc9aca5aa50be7d853c" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 41337 } } }
I20260812 01:18:55.756973 24748 sys_catalog.cc:458] T 00000000000000000000000000000000 P e139560df79e4bc9aca5aa50be7d853c [sys.catalog]: This master's current role is: LEADER
I20260812 01:18:55.757189 24764 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:18:55.756906 24746 sys_catalog.cc:458] T 00000000000000000000000000000000 P e139560df79e4bc9aca5aa50be7d853c [sys.catalog]: This master's current role is: LEADER
I20260812 01:18:55.757805 24764 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:18:55.761111 24764 catalog_manager.cc:1347] Generated new cluster ID: 60595069befc4865b3ccabe2d8b6c051
I20260812 01:18:55.761168 24764 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 01:18:55 dist-test-slave-t3q3 krb5kdc[24588](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497535, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
I20260812 01:18:55.765507 24741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:18:55.749201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41438 (local address 127.24.0.190:41337)
0812 01:18:55.749408 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0812 01:18:55.749418 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 01:18:55.749779 (+ 361us) server_negotiation.cc:407] Connection header received
0812 01:18:55.752590 (+ 2811us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:18:55.752597 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:18:55.752642 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:18:55.752778 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:18:55.754280 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:18:55.755350 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:18:55.756831 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:18:55.757458 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:18:55.757556 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:18:55.761919 (+ 4363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:18:55.761924 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:18:55.761924 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:18:55.761939 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:18:55.763641 (+ 1702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:18:55.764134 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:18:55.764137 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:18:55.764137 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:18:55.764223 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:18:55.764487 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:18:55.764488 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:18:55.764489 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:18:55.764721 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:18:55.764902 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0812 01:18:55.765169 (+ 267us) server_negotiation.cc:299] Negotiation successful
0812 01:18:55.765307 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":86,"threads_started":1}
I20260812 01:18:55.769026 24764 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:18:55.769526 24764 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:18:55.777154 24764 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e139560df79e4bc9aca5aa50be7d853c: Generated new TSK 0
I20260812 01:18:55.777340 24764 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.0.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.0.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.0.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.129.keytab.
Entry for principal kudu/127.24.0.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.129.keytab.
I20260812 01:18:55.810752 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786497534195564-24578-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786497534195564-24578-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786497534195564-24578-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786497534195564-24578-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.0.129:0
--local_ip_for_outbound_sockets=127.24.0.129
--webserver_interface=127.24.0.129
--webserver_port=0
--tserver_master_addrs=127.24.0.190:41337
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.129.keytab
--principal=kudu/127.24.0.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:41753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:18:55.914095 24781 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:18:55.914336 24781 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:18:55.914376 24781 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:18:55.915539 24781 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:18:55.915601 24781 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:18:55.915647 24781 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.0.129
I20260812 01:18:55.917227 24781 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:41753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786497534195564-24578-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786497534195564-24578-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.129.keytab
--principal=kudu/127.24.0.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.0.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786497534195564-24578-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.0.129
--webserver_port=0
--tserver_master_addrs=127.24.0.190:41337
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24781
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.0.129
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786497534195564-24578-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:18:55.917486 24781 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:18:55.917732 24781 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:18:55.922845 24791 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:18:55.922890 24787 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:18:55.922906 24793 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:18:55.923318 24781 server_base.cc:1034] running on GCE node
I20260812 01:18:55.923550 24781 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:18:55.923816 24781 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:18:55.924988 24781 hybrid_clock.cc:648] HybridClock initialized: now 1786497535924971 us; error 36 us; skew 500 ppm
Aug 12 01:18:55 dist-test-slave-t3q3 krb5kdc[24588](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497535, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:18:55.927582 24781 init.cc:377] Logged in from keytab as kudu/127.24.0.129@KRBTEST.COM (short username kudu)
I20260812 01:18:55.928171 24781 webserver.cc:460] Webserver started at http://127.24.0.129:41769/ using document root <none> and password file <none>
I20260812 01:18:55.928406 24781 fs_manager.cc:359] Metadata directory not provided
I20260812 01:18:55.928468 24781 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:18:55.928618 24781 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:18:55.929683 24781 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786497534195564-24578-0/minicluster-data/ts-0/data/instance:
uuid: "874a6bfbe5ce44ce917aa21634e5d3a7"
format_stamp: "Formatted at 2026-08-12 01:18:55 on dist-test-slave-t3q3"
server_key: "a3fde083b4ef10ae30dc277d50f8c397"
server_key_iv: "9f18e2bf83e2b731cef1a05004070038"
server_key_version: "encryptionkey@0"
I20260812 01:18:55.930065 24781 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786497534195564-24578-0/minicluster-data/ts-0/wal/instance:
uuid: "874a6bfbe5ce44ce917aa21634e5d3a7"
format_stamp: "Formatted at 2026-08-12 01:18:55 on dist-test-slave-t3q3"
server_key: "a3fde083b4ef10ae30dc277d50f8c397"
server_key_iv: "9f18e2bf83e2b731cef1a05004070038"
server_key_version: "encryptionkey@0"
I20260812 01:18:55.931435 24781 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 01:18:55.932180 24806 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:18:55.932374 24781 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 01:18:55.932473 24781 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786497534195564-24578-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786497534195564-24578-0/minicluster-data/ts-0/wal
uuid: "874a6bfbe5ce44ce917aa21634e5d3a7"
format_stamp: "Formatted at 2026-08-12 01:18:55 on dist-test-slave-t3q3"
server_key: "a3fde083b4ef10ae30dc277d50f8c397"
server_key_iv: "9f18e2bf83e2b731cef1a05004070038"
server_key_version: "encryptionkey@0"
I20260812 01:18:55.932562 24781 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786497534195564-24578-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786497534195564-24578-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786497534195564-24578-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:18:55.951221 24781 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:18:55.953306 24781 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:18:55.953476 24781 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:18:55.953728 24781 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:18:55.954056 24781 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:18:55.954116 24781 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:18:55.954157 24781 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:18:55.954200 24781 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:18:55.961824 24781 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.129:37299
I20260812 01:18:55.961884 24988 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.129:37299 every 8 connection(s)
I20260812 01:18:55.962225 24781 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786497534195564-24578-0/minicluster-data/ts-0/data/info.pb
I20260812 01:18:55.967645 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 24781
I20260812 01:18:55.967779 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786497534195564-24578-0/minicluster-data/ts-0/wal/instance
I20260812 01:18:55.967949 24578 external_mini_cluster.cc:1428] Setting key 89d7caa99ec53a841af60d577ad2e9bd
Aug 12 01:18:55 dist-test-slave-t3q3 krb5kdc[24588](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497535, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.129@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
I20260812 01:18:55.971988 24991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:18:55.962966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:41337 (local address 127.24.0.129:47997)
0812 01:18:55.963143 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0812 01:18:55.963185 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 01:18:55.963671 (+ 486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:18:55.963883 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:18:55.963886 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:18:55.964563 (+ 677us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:18:55.964769 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:18:55.964773 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:18:55.965449 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:18:55.965450 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:18:55.966235 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:18:55.966237 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:18:55.966593 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:18:55.966594 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:18:55.966697 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:18:55.967132 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:18:55.967140 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:18:55.968877 (+ 1737us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:18:55.970668 (+ 1791us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:18:55.970675 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:18:55.970677 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:18:55.970898 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:18:55.971105 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:18:55.971106 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:18:55.971106 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:18:55.971216 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:18:55.971512 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:18:55.971517 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:18:55.971629 (+ 112us) client_negotiation.cc:769] Sending connection context
0812 01:18:55.971707 (+ 78us) client_negotiation.cc:240] Negotiation successful
0812 01:18:55.971832 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":95,"threads_started":1}
I20260812 01:18:55.972002 24741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:18:55.963008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.129:47997 (local address 127.24.0.190:41337)
0812 01:18:55.963053 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 01:18:55.963055 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:18:55.963256 (+ 201us) server_negotiation.cc:407] Connection header received
0812 01:18:55.963752 (+ 496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:18:55.963753 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:18:55.963762 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:18:55.963821 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:18:55.964851 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:18:55.965377 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:18:55.966306 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:18:55.966510 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:18:55.966565 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:18:55.968993 (+ 2428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:18:55.968994 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:18:55.968995 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:18:55.969001 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:18:55.970572 (+ 1571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:18:55.970967 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:18:55.970968 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:18:55.970968 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:18:55.971036 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:18:55.971294 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:18:55.971295 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:18:55.971295 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:18:55.971443 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:18:55.971509 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 01:18:55.971750 (+ 241us) server_negotiation.cc:299] Negotiation successful
0812 01:18:55.971866 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 01:18:55.972559 24989 heartbeater.cc:347] Connected to a master server at 127.24.0.190:41337
I20260812 01:18:55.972672 24989 heartbeater.cc:464] Registering TS with master...
I20260812 01:18:55.972844 24989 heartbeater.cc:511] Master 127.24.0.190:41337 requested a full tablet report, sending...
I20260812 01:18:55.973366 24671 ts_manager.cc:194] Registered new tserver with Master: 874a6bfbe5ce44ce917aa21634e5d3a7 (127.24.0.129:37299)
I20260812 01:18:55.974020 24671 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.0.129@KRBTEST.COM'} at 127.24.0.129:47997
WARNING: no policy specified for kudu/127.24.0.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.0.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.0.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.130.keytab.
Entry for principal kudu/127.24.0.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.130.keytab.
I20260812 01:18:55.994210 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786497534195564-24578-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786497534195564-24578-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786497534195564-24578-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786497534195564-24578-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.0.130:0
--local_ip_for_outbound_sockets=127.24.0.130
--webserver_interface=127.24.0.130
--webserver_port=0
--tserver_master_addrs=127.24.0.190:41337
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.130.keytab
--principal=kudu/127.24.0.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:41753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:18:56.076851 24995 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:18:56.077162 24995 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:18:56.077205 24995 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:18:56.078405 24995 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:18:56.078485 24995 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:18:56.078533 24995 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.0.130
I20260812 01:18:56.079994 24995 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:41753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786497534195564-24578-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786497534195564-24578-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.130.keytab
--principal=kudu/127.24.0.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.0.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786497534195564-24578-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.0.130
--webserver_port=0
--tserver_master_addrs=127.24.0.190:41337
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24995
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.0.130
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786497534195564-24578-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:18:56.080287 24995 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:18:56.080518 24995 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:18:56.085481 25006 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:18:56.085445 25008 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:18:56.085445 25004 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:18:56.085727 24995 server_base.cc:1034] running on GCE node
I20260812 01:18:56.085917 24995 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:18:56.086194 24995 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:18:56.087350 24995 hybrid_clock.cc:648] HybridClock initialized: now 1786497536087336 us; error 29 us; skew 500 ppm
Aug 12 01:18:56 dist-test-slave-t3q3 krb5kdc[24588](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497536, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:18:56.090003 24995 init.cc:377] Logged in from keytab as kudu/127.24.0.130@KRBTEST.COM (short username kudu)
I20260812 01:18:56.090497 24995 webserver.cc:460] Webserver started at http://127.24.0.130:33295/ using document root <none> and password file <none>
I20260812 01:18:56.090718 24995 fs_manager.cc:359] Metadata directory not provided
I20260812 01:18:56.090765 24995 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:18:56.090934 24995 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:18:56.091996 24995 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786497534195564-24578-0/minicluster-data/ts-1/data/instance:
uuid: "e82ba034742e4897a516ba86c13c37ba"
format_stamp: "Formatted at 2026-08-12 01:18:56 on dist-test-slave-t3q3"
server_key: "739187b04890c837cc7279f899fc7421"
server_key_iv: "09a61eac0b56c96851b57700685b9973"
server_key_version: "encryptionkey@0"
I20260812 01:18:56.092428 24995 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786497534195564-24578-0/minicluster-data/ts-1/wal/instance:
uuid: "e82ba034742e4897a516ba86c13c37ba"
format_stamp: "Formatted at 2026-08-12 01:18:56 on dist-test-slave-t3q3"
server_key: "739187b04890c837cc7279f899fc7421"
server_key_iv: "09a61eac0b56c96851b57700685b9973"
server_key_version: "encryptionkey@0"
I20260812 01:18:56.093921 24995 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 01:18:56.094653 25016 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:18:56.094826 24995 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 01:18:56.094933 24995 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786497534195564-24578-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786497534195564-24578-0/minicluster-data/ts-1/wal
uuid: "e82ba034742e4897a516ba86c13c37ba"
format_stamp: "Formatted at 2026-08-12 01:18:56 on dist-test-slave-t3q3"
server_key: "739187b04890c837cc7279f899fc7421"
server_key_iv: "09a61eac0b56c96851b57700685b9973"
server_key_version: "encryptionkey@0"
I20260812 01:18:56.095007 24995 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786497534195564-24578-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786497534195564-24578-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786497534195564-24578-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:18:56.110246 24995 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:18:56.112685 24995 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:18:56.112856 24995 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:18:56.113183 24995 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:18:56.113565 24995 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:18:56.113618 24995 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:18:56.113662 24995 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:18:56.113703 24995 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:18:56.121968 24995 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.130:41237
I20260812 01:18:56.122020 25193 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.130:41237 every 8 connection(s)
I20260812 01:18:56.122385 24995 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786497534195564-24578-0/minicluster-data/ts-1/data/info.pb
Aug 12 01:18:56 dist-test-slave-t3q3 krb5kdc[24588](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497536, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.130@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
I20260812 01:18:56.129247 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 24995
I20260812 01:18:56.129335 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786497534195564-24578-0/minicluster-data/ts-1/wal/instance
I20260812 01:18:56.129521 24578 external_mini_cluster.cc:1428] Setting key 59bbad9a62bae21de65853d2b3d65e0b
I20260812 01:18:56.132886 24741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:18:56.123420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.130:55199 (local address 127.24.0.190:41337)
0812 01:18:56.123490 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0812 01:18:56.123493 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:18:56.123729 (+ 236us) server_negotiation.cc:407] Connection header received
0812 01:18:56.124307 (+ 578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:18:56.124308 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:18:56.124318 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:18:56.124375 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:18:56.125319 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:18:56.125801 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:18:56.126746 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:18:56.126921 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:18:56.126971 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:18:56.129634 (+ 2663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:18:56.129635 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:18:56.129635 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:18:56.129642 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:18:56.131402 (+ 1760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:18:56.131802 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:18:56.131803 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:18:56.131803 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:18:56.131871 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:18:56.132254 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:18:56.132255 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:18:56.132255 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:18:56.132362 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:18:56.132415 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 01:18:56.132750 (+ 335us) server_negotiation.cc:299] Negotiation successful
0812 01:18:56.132824 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 01:18:56.133170 25196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:18:56.123413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:41337 (local address 127.24.0.130:55199)
0812 01:18:56.123582 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0812 01:18:56.123625 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 01:18:56.124220 (+ 595us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:18:56.124437 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:18:56.124440 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:18:56.125046 (+ 606us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:18:56.125244 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:18:56.125249 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:18:56.125881 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:18:56.125882 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:18:56.126667 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:18:56.126668 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:18:56.126988 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:18:56.126989 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:18:56.127085 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:18:56.127690 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:18:56.127716 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:18:56.129513 (+ 1797us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:18:56.131492 (+ 1979us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:18:56.131497 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:18:56.131499 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:18:56.131733 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:18:56.131957 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:18:56.131958 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:18:56.131959 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:18:56.132139 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:18:56.132426 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:18:56.132432 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:18:56.132602 (+ 170us) client_negotiation.cc:769] Sending connection context
0812 01:18:56.132726 (+ 124us) client_negotiation.cc:240] Negotiation successful
0812 01:18:56.132906 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":92,"threads_started":1}
I20260812 01:18:56.133616 25194 heartbeater.cc:347] Connected to a master server at 127.24.0.190:41337
I20260812 01:18:56.133723 25194 heartbeater.cc:464] Registering TS with master...
I20260812 01:18:56.133916 25194 heartbeater.cc:511] Master 127.24.0.190:41337 requested a full tablet report, sending...
I20260812 01:18:56.134297 24671 ts_manager.cc:194] Registered new tserver with Master: e82ba034742e4897a516ba86c13c37ba (127.24.0.130:41237)
I20260812 01:18:56.134660 24671 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.0.130@KRBTEST.COM'} at 127.24.0.130:55199
WARNING: no policy specified for kudu/127.24.0.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.0.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.0.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.131.keytab.
Entry for principal kudu/127.24.0.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.131.keytab.
I20260812 01:18:56.156426 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786497534195564-24578-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786497534195564-24578-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786497534195564-24578-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786497534195564-24578-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.0.131:0
--local_ip_for_outbound_sockets=127.24.0.131
--webserver_interface=127.24.0.131
--webserver_port=0
--tserver_master_addrs=127.24.0.190:41337
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.131.keytab
--principal=kudu/127.24.0.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:41753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:18:56.241406 25201 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:18:56.241670 25201 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:18:56.241715 25201 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:18:56.242944 25201 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:18:56.243027 25201 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:18:56.243074 25201 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.0.131
I20260812 01:18:56.244643 25201 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:41753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786497534195564-24578-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786497534195564-24578-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.131.keytab
--principal=kudu/127.24.0.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.0.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786497534195564-24578-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.0.131
--webserver_port=0
--tserver_master_addrs=127.24.0.190:41337
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25201
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.0.131
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786497534195564-24578-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:18:56.244932 25201 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:18:56.245194 25201 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:18:56.250726 25208 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:18:56.250726 25211 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:18:56.250747 25207 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:18:56.250797 25201 server_base.cc:1034] running on GCE node
I20260812 01:18:56.251127 25201 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:18:56.251392 25201 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:18:56.252557 25201 hybrid_clock.cc:648] HybridClock initialized: now 1786497536252546 us; error 34 us; skew 500 ppm
Aug 12 01:18:56 dist-test-slave-t3q3 krb5kdc[24588](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497536, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:18:56.255237 25201 init.cc:377] Logged in from keytab as kudu/127.24.0.131@KRBTEST.COM (short username kudu)
I20260812 01:18:56.255702 25201 webserver.cc:460] Webserver started at http://127.24.0.131:42853/ using document root <none> and password file <none>
I20260812 01:18:56.255951 25201 fs_manager.cc:359] Metadata directory not provided
I20260812 01:18:56.256021 25201 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:18:56.256222 25201 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:18:56.257328 25201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786497534195564-24578-0/minicluster-data/ts-2/data/instance:
uuid: "9087e716bec645468812f857cb5e70f3"
format_stamp: "Formatted at 2026-08-12 01:18:56 on dist-test-slave-t3q3"
server_key: "b84a55893f9cf60780c6d0061a773c79"
server_key_iv: "de75d6ccfce57a60d96c0f8c3add14bb"
server_key_version: "encryptionkey@0"
I20260812 01:18:56.257711 25201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786497534195564-24578-0/minicluster-data/ts-2/wal/instance:
uuid: "9087e716bec645468812f857cb5e70f3"
format_stamp: "Formatted at 2026-08-12 01:18:56 on dist-test-slave-t3q3"
server_key: "b84a55893f9cf60780c6d0061a773c79"
server_key_iv: "de75d6ccfce57a60d96c0f8c3add14bb"
server_key_version: "encryptionkey@0"
I20260812 01:18:56.259188 25201 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 01:18:56.259975 25225 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:18:56.260309 25201 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 01:18:56.260416 25201 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786497534195564-24578-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786497534195564-24578-0/minicluster-data/ts-2/wal
uuid: "9087e716bec645468812f857cb5e70f3"
format_stamp: "Formatted at 2026-08-12 01:18:56 on dist-test-slave-t3q3"
server_key: "b84a55893f9cf60780c6d0061a773c79"
server_key_iv: "de75d6ccfce57a60d96c0f8c3add14bb"
server_key_version: "encryptionkey@0"
I20260812 01:18:56.260495 25201 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786497534195564-24578-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786497534195564-24578-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786497534195564-24578-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:18:56.286931 25201 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:18:56.289110 25201 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:18:56.289284 25201 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:18:56.289615 25201 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:18:56.290015 25201 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:18:56.290073 25201 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:18:56.290117 25201 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:18:56.290141 25201 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:18:56.298310 25201 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.131:35523
I20260812 01:18:56.298357 25401 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.131:35523 every 8 connection(s)
I20260812 01:18:56.298776 25201 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786497534195564-24578-0/minicluster-data/ts-2/data/info.pb
I20260812 01:18:56.302709 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 25201
I20260812 01:18:56.302809 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786497534195564-24578-0/minicluster-data/ts-2/wal/instance
I20260812 01:18:56.302982 24578 external_mini_cluster.cc:1428] Setting key 92607fa315b6dc2daaecfa2c305d1653
Aug 12 01:18:56 dist-test-slave-t3q3 krb5kdc[24588](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497536, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.131@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
I20260812 01:18:56.308001 24741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:18:56.299606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.131:52353 (local address 127.24.0.190:41337)
0812 01:18:56.299689 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0812 01:18:56.299691 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:18:56.299891 (+ 200us) server_negotiation.cc:407] Connection header received
0812 01:18:56.300445 (+ 554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:18:56.300446 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:18:56.300457 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:18:56.300514 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:18:56.301486 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:18:56.301973 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:18:56.302986 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:18:56.303259 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:18:56.303348 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:18:56.305610 (+ 2262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:18:56.305611 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:18:56.305611 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:18:56.305617 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:18:56.306817 (+ 1200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:18:56.307173 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:18:56.307173 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:18:56.307174 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:18:56.307225 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:18:56.307490 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:18:56.307491 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:18:56.307491 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:18:56.307588 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:18:56.307638 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 01:18:56.307872 (+ 234us) server_negotiation.cc:299] Negotiation successful
0812 01:18:56.307936 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 01:18:56.308282 25405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:18:56.299565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:41337 (local address 127.24.0.131:52353)
0812 01:18:56.299762 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0812 01:18:56.299800 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 01:18:56.300354 (+ 554us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:18:56.300568 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:18:56.300571 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:18:56.301168 (+ 597us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:18:56.301401 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:18:56.301408 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:18:56.302047 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:18:56.302047 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:18:56.302901 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:18:56.302904 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:18:56.303351 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:18:56.303352 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:18:56.303478 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:18:56.304125 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:18:56.304135 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:18:56.305530 (+ 1395us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:18:56.306886 (+ 1356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:18:56.306892 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:18:56.306894 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:18:56.307109 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:18:56.307313 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:18:56.307314 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:18:56.307314 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:18:56.307427 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:18:56.307642 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:18:56.307646 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:18:56.307761 (+ 115us) client_negotiation.cc:769] Sending connection context
0812 01:18:56.307845 (+ 84us) client_negotiation.cc:240] Negotiation successful
0812 01:18:56.307969 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":108,"threads_started":1}
I20260812 01:18:56.308781 25402 heartbeater.cc:347] Connected to a master server at 127.24.0.190:41337
I20260812 01:18:56.308904 25402 heartbeater.cc:464] Registering TS with master...
I20260812 01:18:56.309130 25402 heartbeater.cc:511] Master 127.24.0.190:41337 requested a full tablet report, sending...
I20260812 01:18:56.309532 24671 ts_manager.cc:194] Registered new tserver with Master: 9087e716bec645468812f857cb5e70f3 (127.24.0.131:35523)
I20260812 01:18:56.309906 24671 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.0.131@KRBTEST.COM'} at 127.24.0.131:52353
I20260812 01:18:56.315078 24578 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 01:18:56 dist-test-slave-t3q3 krb5kdc[24588](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497536, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 01:18:56 dist-test-slave-t3q3 krb5kdc[24588](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497536, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.24.0.129@KRBTEST.COM
I20260812 01:18:56.341488 25414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:18:56.333736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44350 (local address 127.24.0.129:37299)
0812 01:18:56.333922 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0812 01:18:56.333927 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 01:18:56.333939 (+ 12us) server_negotiation.cc:407] Connection header received
0812 01:18:56.334000 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:18:56.334002 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:18:56.334016 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:18:56.334082 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:18:56.334683 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:18:56.335423 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:18:56.336269 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:18:56.336596 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:18:56.336655 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:18:56.338934 (+ 2279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:18:56.338942 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:18:56.338942 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:18:56.338950 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:18:56.340433 (+ 1483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:18:56.340758 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:18:56.340761 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:18:56.340761 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:18:56.340830 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:18:56.340999 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:18:56.341000 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:18:56.341000 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:18:56.341126 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:18:56.341252 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0812 01:18:56.341306 (+ 54us) server_negotiation.cc:299] Negotiation successful
0812 01:18:56.341371 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":98,"threads_started":1}
W20260812 01:18:56.341905 24845 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44350
Aug 12 01:18:56 dist-test-slave-t3q3 krb5kdc[24588](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497536, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 01:18:56 dist-test-slave-t3q3 krb5kdc[24588](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497536, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.0.129@KRBTEST.COM
I20260812 01:18:56.366734 25414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:18:56.359485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44354 (local address 127.24.0.129:37299)
0812 01:18:56.359554 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0812 01:18:56.359556 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:18:56.359638 (+ 82us) server_negotiation.cc:407] Connection header received
0812 01:18:56.359738 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:18:56.359739 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:18:56.359756 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:18:56.359820 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:18:56.360509 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:18:56.361007 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:18:56.361757 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:18:56.361963 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:18:56.362034 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:18:56.364414 (+ 2380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:18:56.364416 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:18:56.364416 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:18:56.364423 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:18:56.365680 (+ 1257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:18:56.366055 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:18:56.366056 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:18:56.366056 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:18:56.366106 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:18:56.366310 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:18:56.366311 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:18:56.366311 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:18:56.366401 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:18:56.366491 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 01:18:56.366563 (+ 72us) server_negotiation.cc:299] Negotiation successful
0812 01:18:56.366628 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 01:18:56.367327 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 24781
I20260812 01:18:56.382180 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 24995
I20260812 01:18:56.390457 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 25201
I20260812 01:18:56.398152 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 24612
2026-08-12T01:18:56Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2215 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786497534195564-24578-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 01:18:56 dist-test-slave-t3q3 krb5kdc[25421](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 01:18:56 dist-test-slave-t3q3 krb5kdc[25421](info): set up 2 sockets
Aug 12 01:18:56 dist-test-slave-t3q3 krb5kdc[25421](info): commencing operation
krb5kdc: starting...
W20260812 01:18:57.693470 24578 mini_kdc.cc:121] Time spent starting KDC: real 1.269s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786497534195564-24578-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786497534195564-24578-0/krb5kdc/test-user.keytab.
Aug 12 01:18:57 dist-test-slave-t3q3 krb5kdc[25421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497537, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T01:18:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:18:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.0.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.0.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.0.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab.
Entry for principal kudu/127.24.0.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab.
I20260812 01:18:57.817101 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786497534195564-24578-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786497534195564-24578-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786497534195564-24578-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.190:39711
--webserver_interface=127.24.0.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab
--principal=kudu/127.24.0.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:39511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.0.190:39711
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:18:57.899984 25441 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:18:57.900210 25441 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:18:57.900233 25441 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:18:57.901331 25441 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:18:57.901366 25441 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:18:57.901381 25441 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:18:57.901393 25441 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:18:57.901405 25441 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:18:57.902908 25441 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:39511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786497534195564-24578-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786497534195564-24578-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.0.190:39711
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab
--principal=kudu/127.24.0.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.190:39711
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.0.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25441
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786497534195564-24578-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:18:57.903100 25441 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:18:57.903260 25441 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:18:57.907958 25456 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:18:57.907946 25452 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:18:57.908221 25441 server_base.cc:1034] running on GCE node
W20260812 01:18:57.907943 25453 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:18:57.908515 25441 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:18:57.908792 25441 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:18:57.909934 25441 hybrid_clock.cc:648] HybridClock initialized: now 1786497537909924 us; error 28 us; skew 500 ppm
Aug 12 01:18:57 dist-test-slave-t3q3 krb5kdc[25421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497537, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:18:57.916026 25441 init.cc:377] Logged in from keytab as kudu/127.24.0.190@KRBTEST.COM (short username kudu)
I20260812 01:18:57.916513 25441 webserver.cc:460] Webserver started at http://127.24.0.190:33207/ using document root <none> and password file <none>
I20260812 01:18:57.916720 25441 fs_manager.cc:359] Metadata directory not provided
I20260812 01:18:57.916762 25441 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:18:57.916849 25441 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:18:57.917861 25441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786497534195564-24578-0/minicluster-data/master-0/data/instance:
uuid: "385567e9f8874be58f86bffdbc794681"
format_stamp: "Formatted at 2026-08-12 01:18:57 on dist-test-slave-t3q3"
server_key: "b8dcf68814f9c3f74b37e43240f9baf5"
server_key_iv: "5e42137dc287460cb42d8bc4458e8145"
server_key_version: "encryptionkey@0"
I20260812 01:18:57.918162 25441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786497534195564-24578-0/minicluster-data/master-0/wal/instance:
uuid: "385567e9f8874be58f86bffdbc794681"
format_stamp: "Formatted at 2026-08-12 01:18:57 on dist-test-slave-t3q3"
server_key: "b8dcf68814f9c3f74b37e43240f9baf5"
server_key_iv: "5e42137dc287460cb42d8bc4458e8145"
server_key_version: "encryptionkey@0"
I20260812 01:18:57.919456 25441 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 01:18:57.920208 25468 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:18:57.920468 25441 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 01:18:57.920557 25441 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786497534195564-24578-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786497534195564-24578-0/minicluster-data/master-0/wal
uuid: "385567e9f8874be58f86bffdbc794681"
format_stamp: "Formatted at 2026-08-12 01:18:57 on dist-test-slave-t3q3"
server_key: "b8dcf68814f9c3f74b37e43240f9baf5"
server_key_iv: "5e42137dc287460cb42d8bc4458e8145"
server_key_version: "encryptionkey@0"
I20260812 01:18:57.920614 25441 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786497534195564-24578-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786497534195564-24578-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786497534195564-24578-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:18:57.931348 25441 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:18:57.933306 25441 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:18:57.933419 25441 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:18:57.937245 25441 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.190:39711
I20260812 01:18:57.937292 25546 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.190:39711 every 8 connection(s)
I20260812 01:18:57.937631 25441 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
I20260812 01:18:57.938481 25548 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:18:57.940779 25548 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 385567e9f8874be58f86bffdbc794681: Bootstrap starting.
I20260812 01:18:57.941354 25548 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 385567e9f8874be58f86bffdbc794681: Neither blocks nor log segments found. Creating new log.
I20260812 01:18:57.941648 25548 log.cc:824] T 00000000000000000000000000000000 P 385567e9f8874be58f86bffdbc794681: Log is configured to *not* fsync() on all Append() calls
I20260812 01:18:57.942387 25548 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 385567e9f8874be58f86bffdbc794681: No bootstrap required, opened a new log
I20260812 01:18:57.942621 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 25441
I20260812 01:18:57.942695 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786497534195564-24578-0/minicluster-data/master-0/wal/instance
I20260812 01:18:57.942852 24578 external_mini_cluster.cc:1428] Setting key 92f6dca23ed3e9dd611dce186ad390df
I20260812 01:18:57.944406 25548 raft_consensus.cc:348] T 00000000000000000000000000000000 P 385567e9f8874be58f86bffdbc794681 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "385567e9f8874be58f86bffdbc794681" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 39711 } }
I20260812 01:18:57.944551 25548 raft_consensus.cc:374] T 00000000000000000000000000000000 P 385567e9f8874be58f86bffdbc794681 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:18:57.944597 25548 raft_consensus.cc:729] T 00000000000000000000000000000000 P 385567e9f8874be58f86bffdbc794681 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 385567e9f8874be58f86bffdbc794681, State: Initialized, Role: FOLLOWER
I20260812 01:18:57.944757 25548 consensus_queue.cc:260] T 00000000000000000000000000000000 P 385567e9f8874be58f86bffdbc794681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "385567e9f8874be58f86bffdbc794681" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 39711 } }
I20260812 01:18:57.944844 25548 raft_consensus.cc:388] T 00000000000000000000000000000000 P 385567e9f8874be58f86bffdbc794681 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:18:57.944885 25548 raft_consensus.cc:482] T 00000000000000000000000000000000 P 385567e9f8874be58f86bffdbc794681 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:18:57.944934 25548 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 385567e9f8874be58f86bffdbc794681 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:18:57.945708 25548 raft_consensus.cc:504] T 00000000000000000000000000000000 P 385567e9f8874be58f86bffdbc794681 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "385567e9f8874be58f86bffdbc794681" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 39711 } }
I20260812 01:18:57.945812 25548 leader_election.cc:302] T 00000000000000000000000000000000 P 385567e9f8874be58f86bffdbc794681 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 385567e9f8874be58f86bffdbc794681; no voters:
I20260812 01:18:57.945971 25548 leader_election.cc:288] T 00000000000000000000000000000000 P 385567e9f8874be58f86bffdbc794681 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 01:18:57.946017 25555 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 385567e9f8874be58f86bffdbc794681 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:18:57.946172 25555 raft_consensus.cc:686] T 00000000000000000000000000000000 P 385567e9f8874be58f86bffdbc794681 [term 1 LEADER]: Becoming Leader. State: Replica: 385567e9f8874be58f86bffdbc794681, State: Running, Role: LEADER
I20260812 01:18:57.946209 25548 sys_catalog.cc:564] T 00000000000000000000000000000000 P 385567e9f8874be58f86bffdbc794681 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:18:57.946273 25555 consensus_queue.cc:237] T 00000000000000000000000000000000 P 385567e9f8874be58f86bffdbc794681 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "385567e9f8874be58f86bffdbc794681" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 39711 } }
I20260812 01:18:57.946558 25555 sys_catalog.cc:455] T 00000000000000000000000000000000 P 385567e9f8874be58f86bffdbc794681 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 385567e9f8874be58f86bffdbc794681. Latest consensus state: current_term: 1 leader_uuid: "385567e9f8874be58f86bffdbc794681" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "385567e9f8874be58f86bffdbc794681" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 39711 } } }
I20260812 01:18:57.946653 25555 sys_catalog.cc:458] T 00000000000000000000000000000000 P 385567e9f8874be58f86bffdbc794681 [sys.catalog]: This master's current role is: LEADER
I20260812 01:18:57.946571 25557 sys_catalog.cc:455] T 00000000000000000000000000000000 P 385567e9f8874be58f86bffdbc794681 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "385567e9f8874be58f86bffdbc794681" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "385567e9f8874be58f86bffdbc794681" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 39711 } } }
I20260812 01:18:57.946838 25557 sys_catalog.cc:458] T 00000000000000000000000000000000 P 385567e9f8874be58f86bffdbc794681 [sys.catalog]: This master's current role is: LEADER
I20260812 01:18:57.946975 25563 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:18:57.947458 25563 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:18:57.949594 25563 catalog_manager.cc:1347] Generated new cluster ID: d5977345f44648bcbbbf428c5b25c9ed
I20260812 01:18:57.949656 25563 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 01:18:57 dist-test-slave-t3q3 krb5kdc[25421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497537, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
I20260812 01:18:57.955297 25554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:18:57.943344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46018 (local address 127.24.0.190:39711)
0812 01:18:57.943553 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0812 01:18:57.943560 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 01:18:57.943589 (+ 29us) server_negotiation.cc:407] Connection header received
0812 01:18:57.944032 (+ 443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:18:57.944038 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:18:57.944142 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:18:57.944281 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:18:57.944971 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:18:57.946771 (+ 1800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:18:57.947984 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:18:57.948604 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:18:57.948724 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:18:57.951446 (+ 2722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:18:57.951449 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:18:57.951450 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:18:57.951461 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:18:57.953482 (+ 2021us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:18:57.953901 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:18:57.953904 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:18:57.953905 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:18:57.954014 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:18:57.954237 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:18:57.954238 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:18:57.954238 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:18:57.954523 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:18:57.954693 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0812 01:18:57.954985 (+ 292us) server_negotiation.cc:299] Negotiation successful
0812 01:18:57.955127 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":93,"threads_started":1}
I20260812 01:18:57.955390 25563 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:18:57.956015 25563 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:18:57.960424 25563 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 385567e9f8874be58f86bffdbc794681: Generated new TSK 0
I20260812 01:18:57.960592 25563 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.0.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.0.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.0.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.129.keytab.
Entry for principal kudu/127.24.0.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.129.keytab.
I20260812 01:18:57.991070 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786497534195564-24578-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786497534195564-24578-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786497534195564-24578-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786497534195564-24578-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.0.129:0
--local_ip_for_outbound_sockets=127.24.0.129
--webserver_interface=127.24.0.129
--webserver_port=0
--tserver_master_addrs=127.24.0.190:39711
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.129.keytab
--principal=kudu/127.24.0.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:39511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:18:58.074939 25590 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:18:58.075145 25590 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:18:58.075170 25590 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:18:58.076393 25590 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:18:58.076435 25590 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:18:58.076463 25590 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.0.129
I20260812 01:18:58.077966 25590 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:39511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786497534195564-24578-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786497534195564-24578-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.129.keytab
--principal=kudu/127.24.0.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.0.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786497534195564-24578-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.0.129
--webserver_port=0
--tserver_master_addrs=127.24.0.190:39711
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25590
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.0.129
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786497534195564-24578-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:18:58.078229 25590 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:18:58.078449 25590 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:18:58.083439 25596 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:18:58.083453 25601 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:18:58.083460 25597 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:18:58.083657 25590 server_base.cc:1034] running on GCE node
I20260812 01:18:58.083881 25590 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:18:58.084163 25590 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:18:58.085301 25590 hybrid_clock.cc:648] HybridClock initialized: now 1786497538085290 us; error 26 us; skew 500 ppm
Aug 12 01:18:58 dist-test-slave-t3q3 krb5kdc[25421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497538, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:18:58.087845 25590 init.cc:377] Logged in from keytab as kudu/127.24.0.129@KRBTEST.COM (short username kudu)
I20260812 01:18:58.088392 25590 webserver.cc:460] Webserver started at http://127.24.0.129:37523/ using document root <none> and password file <none>
I20260812 01:18:58.088624 25590 fs_manager.cc:359] Metadata directory not provided
I20260812 01:18:58.088668 25590 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:18:58.088802 25590 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:18:58.089808 25590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786497534195564-24578-0/minicluster-data/ts-0/data/instance:
uuid: "b5bcf4602b864e5b9758131bedb6a75d"
format_stamp: "Formatted at 2026-08-12 01:18:58 on dist-test-slave-t3q3"
server_key: "9a23eeb107f0c0f00845c258b076fbf3"
server_key_iv: "731e1dcb3a3200aa1bff1d315e8624b3"
server_key_version: "encryptionkey@0"
I20260812 01:18:58.090153 25590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786497534195564-24578-0/minicluster-data/ts-0/wal/instance:
uuid: "b5bcf4602b864e5b9758131bedb6a75d"
format_stamp: "Formatted at 2026-08-12 01:18:58 on dist-test-slave-t3q3"
server_key: "9a23eeb107f0c0f00845c258b076fbf3"
server_key_iv: "731e1dcb3a3200aa1bff1d315e8624b3"
server_key_version: "encryptionkey@0"
I20260812 01:18:58.091460 25590 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 01:18:58.092250 25613 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:18:58.092442 25590 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:18:58.092530 25590 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786497534195564-24578-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786497534195564-24578-0/minicluster-data/ts-0/wal
uuid: "b5bcf4602b864e5b9758131bedb6a75d"
format_stamp: "Formatted at 2026-08-12 01:18:58 on dist-test-slave-t3q3"
server_key: "9a23eeb107f0c0f00845c258b076fbf3"
server_key_iv: "731e1dcb3a3200aa1bff1d315e8624b3"
server_key_version: "encryptionkey@0"
I20260812 01:18:58.092599 25590 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786497534195564-24578-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786497534195564-24578-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786497534195564-24578-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:18:58.110384 25590 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:18:58.112778 25590 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:18:58.112937 25590 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:18:58.113175 25590 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:18:58.113499 25590 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:18:58.113533 25590 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:18:58.113572 25590 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:18:58.113600 25590 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:18:58.119915 25590 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.129:41267
I20260812 01:18:58.119987 25779 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.129:41267 every 8 connection(s)
I20260812 01:18:58.120339 25590 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786497534195564-24578-0/minicluster-data/ts-0/data/info.pb
Aug 12 01:18:58 dist-test-slave-t3q3 krb5kdc[25421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497538, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.129@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
I20260812 01:18:58.126888 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 25590
I20260812 01:18:58.126983 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786497534195564-24578-0/minicluster-data/ts-0/wal/instance
I20260812 01:18:58.127139 24578 external_mini_cluster.cc:1428] Setting key b009c49b2ddaeada226fe8729a5cd1d9
I20260812 01:18:58.130041 25554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:18:58.121068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.129:34995 (local address 127.24.0.190:39711)
0812 01:18:58.121129 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 01:18:58.121132 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:18:58.121372 (+ 240us) server_negotiation.cc:407] Connection header received
0812 01:18:58.122431 (+ 1059us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:18:58.122432 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:18:58.122441 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:18:58.122486 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:18:58.123140 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:18:58.123697 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:18:58.124823 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:18:58.125021 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:18:58.125076 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:18:58.127122 (+ 2046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:18:58.127124 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:18:58.127124 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:18:58.127131 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:18:58.128639 (+ 1508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:18:58.129056 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:18:58.129058 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:18:58.129058 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:18:58.129120 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:18:58.129394 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:18:58.129395 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:18:58.129395 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:18:58.129506 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:18:58.129576 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 01:18:58.129854 (+ 278us) server_negotiation.cc:299] Negotiation successful
0812 01:18:58.129949 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 01:18:58.130223 25782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:18:58.121047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:39711 (local address 127.24.0.129:34995)
0812 01:18:58.121244 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0812 01:18:58.121286 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 01:18:58.122319 (+ 1033us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:18:58.122541 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:18:58.122544 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:18:58.122860 (+ 316us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:18:58.123061 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:18:58.123066 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:18:58.123787 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:18:58.123788 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:18:58.124749 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:18:58.124752 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:18:58.125094 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:18:58.125095 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:18:58.125207 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:18:58.125647 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:18:58.125655 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:18:58.127038 (+ 1383us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:18:58.128740 (+ 1702us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:18:58.128748 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:18:58.128750 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:18:58.128978 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:18:58.129203 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:18:58.129203 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:18:58.129204 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:18:58.129317 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:18:58.129580 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:18:58.129587 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:18:58.129727 (+ 140us) client_negotiation.cc:769] Sending connection context
0812 01:18:58.129833 (+ 106us) client_negotiation.cc:240] Negotiation successful
0812 01:18:58.129986 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":98,"threads_started":1}
I20260812 01:18:58.130715 25780 heartbeater.cc:347] Connected to a master server at 127.24.0.190:39711
I20260812 01:18:58.130826 25780 heartbeater.cc:464] Registering TS with master...
I20260812 01:18:58.131007 25780 heartbeater.cc:511] Master 127.24.0.190:39711 requested a full tablet report, sending...
I20260812 01:18:58.131560 25489 ts_manager.cc:194] Registered new tserver with Master: b5bcf4602b864e5b9758131bedb6a75d (127.24.0.129:41267)
I20260812 01:18:58.132568 25489 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.0.129@KRBTEST.COM'} at 127.24.0.129:34995
WARNING: no policy specified for kudu/127.24.0.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.0.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.0.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.130.keytab.
Entry for principal kudu/127.24.0.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.130.keytab.
I20260812 01:18:58.154560 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786497534195564-24578-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786497534195564-24578-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786497534195564-24578-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786497534195564-24578-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.0.130:0
--local_ip_for_outbound_sockets=127.24.0.130
--webserver_interface=127.24.0.130
--webserver_port=0
--tserver_master_addrs=127.24.0.190:39711
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.130.keytab
--principal=kudu/127.24.0.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:39511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:18:58.237974 25788 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:18:58.238170 25788 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:18:58.238194 25788 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:18:58.239269 25788 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:18:58.239301 25788 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:18:58.239326 25788 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.0.130
I20260812 01:18:58.240795 25788 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:39511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786497534195564-24578-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786497534195564-24578-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.130.keytab
--principal=kudu/127.24.0.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.0.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786497534195564-24578-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.0.130
--webserver_port=0
--tserver_master_addrs=127.24.0.190:39711
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25788
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.0.130
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786497534195564-24578-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:18:58.240983 25788 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:18:58.241153 25788 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:18:58.246146 25799 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:18:58.246145 25797 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:18:58.246364 25794 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:18:58.246340 25788 server_base.cc:1034] running on GCE node
I20260812 01:18:58.246727 25788 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:18:58.246991 25788 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:18:58.248116 25788 hybrid_clock.cc:648] HybridClock initialized: now 1786497538248099 us; error 28 us; skew 500 ppm
Aug 12 01:18:58 dist-test-slave-t3q3 krb5kdc[25421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497538, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:18:58.250753 25788 init.cc:377] Logged in from keytab as kudu/127.24.0.130@KRBTEST.COM (short username kudu)
I20260812 01:18:58.251180 25788 webserver.cc:460] Webserver started at http://127.24.0.130:39703/ using document root <none> and password file <none>
I20260812 01:18:58.251386 25788 fs_manager.cc:359] Metadata directory not provided
I20260812 01:18:58.251426 25788 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:18:58.251513 25788 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:18:58.252540 25788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786497534195564-24578-0/minicluster-data/ts-1/data/instance:
uuid: "22343a784b424372b7677f7f66581e51"
format_stamp: "Formatted at 2026-08-12 01:18:58 on dist-test-slave-t3q3"
server_key: "bd272c71ce87956ea089354f3a17498a"
server_key_iv: "e81bede034262b38131730703b81e25e"
server_key_version: "encryptionkey@0"
I20260812 01:18:58.252841 25788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786497534195564-24578-0/minicluster-data/ts-1/wal/instance:
uuid: "22343a784b424372b7677f7f66581e51"
format_stamp: "Formatted at 2026-08-12 01:18:58 on dist-test-slave-t3q3"
server_key: "bd272c71ce87956ea089354f3a17498a"
server_key_iv: "e81bede034262b38131730703b81e25e"
server_key_version: "encryptionkey@0"
I20260812 01:18:58.254096 25788 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 01:18:58.254769 25810 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:18:58.254951 25788 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 01:18:58.255034 25788 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786497534195564-24578-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786497534195564-24578-0/minicluster-data/ts-1/wal
uuid: "22343a784b424372b7677f7f66581e51"
format_stamp: "Formatted at 2026-08-12 01:18:58 on dist-test-slave-t3q3"
server_key: "bd272c71ce87956ea089354f3a17498a"
server_key_iv: "e81bede034262b38131730703b81e25e"
server_key_version: "encryptionkey@0"
I20260812 01:18:58.255088 25788 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786497534195564-24578-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786497534195564-24578-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786497534195564-24578-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:18:58.273739 25788 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:18:58.275646 25788 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:18:58.275769 25788 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:18:58.275969 25788 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:18:58.276342 25788 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:18:58.276377 25788 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:18:58.276432 25788 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:18:58.276463 25788 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:18:58.282653 25788 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.130:44769
I20260812 01:18:58.282733 25980 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.130:44769 every 8 connection(s)
I20260812 01:18:58.283061 25788 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786497534195564-24578-0/minicluster-data/ts-1/data/info.pb
Aug 12 01:18:58 dist-test-slave-t3q3 krb5kdc[25421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497538, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.130@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
I20260812 01:18:58.290318 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 25788
I20260812 01:18:58.290413 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786497534195564-24578-0/minicluster-data/ts-1/wal/instance
I20260812 01:18:58.290588 24578 external_mini_cluster.cc:1428] Setting key 970d065be4adbf448aa31f65103d63a0
I20260812 01:18:58.292729 25554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:18:58.283855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.130:49741 (local address 127.24.0.190:39711)
0812 01:18:58.283901 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 01:18:58.283903 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:18:58.284122 (+ 219us) server_negotiation.cc:407] Connection header received
0812 01:18:58.285103 (+ 981us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:18:58.285104 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:18:58.285117 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:18:58.285189 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:18:58.285799 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:18:58.286328 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:18:58.287275 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:18:58.287457 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:18:58.287508 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:18:58.289670 (+ 2162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:18:58.289672 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:18:58.289673 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:18:58.289681 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:18:58.291281 (+ 1600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:18:58.291725 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:18:58.291726 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:18:58.291727 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:18:58.291788 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:18:58.292041 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:18:58.292042 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:18:58.292042 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:18:58.292183 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:18:58.292261 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 01:18:58.292549 (+ 288us) server_negotiation.cc:299] Negotiation successful
0812 01:18:58.292636 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 01:18:58.292963 25986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:18:58.283814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:39711 (local address 127.24.0.130:49741)
0812 01:18:58.283988 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0812 01:18:58.284029 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 01:18:58.284980 (+ 951us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:18:58.285265 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:18:58.285269 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:18:58.285525 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:18:58.285722 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:18:58.285726 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:18:58.286400 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:18:58.286401 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:18:58.287195 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:18:58.287206 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:18:58.287525 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:18:58.287526 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:18:58.287632 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:18:58.288179 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:18:58.288187 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:18:58.289554 (+ 1367us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:18:58.291380 (+ 1826us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:18:58.291388 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:18:58.291391 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:18:58.291630 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:18:58.291855 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:18:58.291856 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:18:58.291856 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:18:58.291969 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:18:58.292253 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:18:58.292260 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:18:58.292401 (+ 141us) client_negotiation.cc:769] Sending connection context
0812 01:18:58.292524 (+ 123us) client_negotiation.cc:240] Negotiation successful
0812 01:18:58.292683 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":94,"threads_started":1}
I20260812 01:18:58.293474 25981 heartbeater.cc:347] Connected to a master server at 127.24.0.190:39711
I20260812 01:18:58.293563 25981 heartbeater.cc:464] Registering TS with master...
I20260812 01:18:58.293752 25981 heartbeater.cc:511] Master 127.24.0.190:39711 requested a full tablet report, sending...
I20260812 01:18:58.294147 25489 ts_manager.cc:194] Registered new tserver with Master: 22343a784b424372b7677f7f66581e51 (127.24.0.130:44769)
I20260812 01:18:58.294646 25489 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.0.130@KRBTEST.COM'} at 127.24.0.130:49741
WARNING: no policy specified for kudu/127.24.0.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.0.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.0.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.131.keytab.
Entry for principal kudu/127.24.0.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.131.keytab.
I20260812 01:18:58.316288 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786497534195564-24578-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786497534195564-24578-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786497534195564-24578-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786497534195564-24578-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.0.131:0
--local_ip_for_outbound_sockets=127.24.0.131
--webserver_interface=127.24.0.131
--webserver_port=0
--tserver_master_addrs=127.24.0.190:39711
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.131.keytab
--principal=kudu/127.24.0.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:39511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:18:58.402446 25990 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:18:58.402650 25990 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:18:58.402674 25990 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:18:58.403745 25990 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:18:58.403780 25990 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:18:58.403806 25990 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.0.131
I20260812 01:18:58.405257 25990 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:39511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786497534195564-24578-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786497534195564-24578-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.131.keytab
--principal=kudu/127.24.0.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.0.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786497534195564-24578-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.0.131
--webserver_port=0
--tserver_master_addrs=127.24.0.190:39711
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25990
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.0.131
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786497534195564-24578-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:18:58.405445 25990 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:18:58.405624 25990 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:18:58.410544 25999 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:18:58.410558 26002 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:18:58.410498 25997 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:18:58.410808 25990 server_base.cc:1034] running on GCE node
I20260812 01:18:58.411031 25990 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:18:58.411273 25990 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:18:58.412411 25990 hybrid_clock.cc:648] HybridClock initialized: now 1786497538412400 us; error 29 us; skew 500 ppm
Aug 12 01:18:58 dist-test-slave-t3q3 krb5kdc[25421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497538, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:18:58.414990 25990 init.cc:377] Logged in from keytab as kudu/127.24.0.131@KRBTEST.COM (short username kudu)
I20260812 01:18:58.415421 25990 webserver.cc:460] Webserver started at http://127.24.0.131:40321/ using document root <none> and password file <none>
I20260812 01:18:58.415663 25990 fs_manager.cc:359] Metadata directory not provided
I20260812 01:18:58.415729 25990 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:18:58.415879 25990 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:18:58.416941 25990 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786497534195564-24578-0/minicluster-data/ts-2/data/instance:
uuid: "b104dce7b86f4602a527379ea53ae301"
format_stamp: "Formatted at 2026-08-12 01:18:58 on dist-test-slave-t3q3"
server_key: "b3598c56beadbc9b0521bf39c4d000dc"
server_key_iv: "9c8d40da6969ee90ff9c00a50c825253"
server_key_version: "encryptionkey@0"
I20260812 01:18:58.417302 25990 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786497534195564-24578-0/minicluster-data/ts-2/wal/instance:
uuid: "b104dce7b86f4602a527379ea53ae301"
format_stamp: "Formatted at 2026-08-12 01:18:58 on dist-test-slave-t3q3"
server_key: "b3598c56beadbc9b0521bf39c4d000dc"
server_key_iv: "9c8d40da6969ee90ff9c00a50c825253"
server_key_version: "encryptionkey@0"
I20260812 01:18:58.418697 25990 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 01:18:58.419425 26013 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:18:58.419596 25990 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 01:18:58.419693 25990 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786497534195564-24578-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786497534195564-24578-0/minicluster-data/ts-2/wal
uuid: "b104dce7b86f4602a527379ea53ae301"
format_stamp: "Formatted at 2026-08-12 01:18:58 on dist-test-slave-t3q3"
server_key: "b3598c56beadbc9b0521bf39c4d000dc"
server_key_iv: "9c8d40da6969ee90ff9c00a50c825253"
server_key_version: "encryptionkey@0"
I20260812 01:18:58.419777 25990 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786497534195564-24578-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786497534195564-24578-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786497534195564-24578-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:18:58.440229 25990 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:18:58.442760 25990 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:18:58.442932 25990 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:18:58.443184 25990 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:18:58.443518 25990 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:18:58.443571 25990 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:18:58.443608 25990 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:18:58.443650 25990 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:18:58.450273 25990 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.131:38317
I20260812 01:18:58.450344 26206 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.131:38317 every 8 connection(s)
I20260812 01:18:58.450683 25990 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786497534195564-24578-0/minicluster-data/ts-2/data/info.pb
I20260812 01:18:58.451612 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 25990
I20260812 01:18:58.451730 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786497534195564-24578-0/minicluster-data/ts-2/wal/instance
I20260812 01:18:58.451928 24578 external_mini_cluster.cc:1428] Setting key 9973a67c948796b12f0b9513eefa2af6
Aug 12 01:18:58 dist-test-slave-t3q3 krb5kdc[25421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497538, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.131@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
I20260812 01:18:58.460598 25554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:18:58.452132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.131:49929 (local address 127.24.0.190:39711)
0812 01:18:58.452187 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0812 01:18:58.452189 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:18:58.452203 (+ 14us) server_negotiation.cc:407] Connection header received
0812 01:18:58.453126 (+ 923us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:18:58.453126 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:18:58.453136 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:18:58.453183 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:18:58.453763 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:18:58.454250 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:18:58.455202 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:18:58.455513 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:18:58.455572 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:18:58.457770 (+ 2198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:18:58.457771 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:18:58.457772 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:18:58.457778 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:18:58.459028 (+ 1250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:18:58.459599 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:18:58.459600 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:18:58.459601 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:18:58.459672 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:18:58.459896 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:18:58.459897 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:18:58.459897 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:18:58.460005 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:18:58.460113 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0812 01:18:58.460423 (+ 310us) server_negotiation.cc:299] Negotiation successful
0812 01:18:58.460520 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 01:18:58.460767 26209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:18:58.451500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:39711 (local address 127.24.0.131:49929)
0812 01:18:58.451676 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0812 01:18:58.451732 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0812 01:18:58.453040 (+ 1308us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:18:58.453240 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:18:58.453244 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:18:58.453498 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:18:58.453693 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:18:58.453698 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:18:58.454327 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:18:58.454328 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:18:58.455125 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:18:58.455126 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:18:58.455590 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:18:58.455590 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:18:58.455692 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:18:58.456260 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:18:58.456269 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:18:58.457670 (+ 1401us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:18:58.459094 (+ 1424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:18:58.459100 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:18:58.459102 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:18:58.459365 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:18:58.459739 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:18:58.459740 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:18:58.459740 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:18:58.459840 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:18:58.460093 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:18:58.460099 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:18:58.460249 (+ 150us) client_negotiation.cc:769] Sending connection context
0812 01:18:58.460366 (+ 117us) client_negotiation.cc:240] Negotiation successful
0812 01:18:58.460542 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":72,"threads_started":1}
I20260812 01:18:58.461220 26207 heartbeater.cc:347] Connected to a master server at 127.24.0.190:39711
I20260812 01:18:58.461339 26207 heartbeater.cc:464] Registering TS with master...
I20260812 01:18:58.461519 26207 heartbeater.cc:511] Master 127.24.0.190:39711 requested a full tablet report, sending...
I20260812 01:18:58.461903 25489 ts_manager.cc:194] Registered new tserver with Master: b104dce7b86f4602a527379ea53ae301 (127.24.0.131:38317)
I20260812 01:18:58.462273 25489 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.0.131@KRBTEST.COM'} at 127.24.0.131:49929
I20260812 01:18:58.463562 24578 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 01:18:58.470969 25554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:18:58.465091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46022 (local address 127.24.0.190:39711)
0812 01:18:58.465149 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0812 01:18:58.465151 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:18:58.465230 (+ 79us) server_negotiation.cc:407] Connection header received
0812 01:18:58.465313 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:18:58.465314 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:18:58.465323 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:18:58.465369 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:18:58.465919 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:18:58.466419 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:18:58.467119 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:18:58.467328 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:18:58.467401 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:18:58.468570 (+ 1169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:18:58.468571 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:18:58.468572 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:18:58.468578 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:18:58.469778 (+ 1200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:18:58.470107 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:18:58.470108 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:18:58.470108 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:18:58.470184 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:18:58.470391 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:18:58.470392 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:18:58.470392 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:18:58.470538 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:18:58.470641 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0812 01:18:58.470788 (+ 147us) server_negotiation.cc:299] Negotiation successful
0812 01:18:58.470870 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260812 01:18:58.473037 25489 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46022:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 01:18:58.474582 25489 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 01:18:58.481405 26221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:18:58.477022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.130:44769 (local address 127.0.0.1:57620)
0812 01:18:58.477336 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0812 01:18:58.477379 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 01:18:58.477450 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:18:58.477673 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:18:58.477676 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:18:58.477678 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:18:58.477749 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:18:58.477751 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:18:58.478911 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:18:58.478912 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:18:58.480234 (+ 1322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:18:58.480237 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:18:58.481068 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:18:58.481069 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:18:58.481162 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:18:58.481178 (+ 16us) client_negotiation.cc:769] Sending connection context
0812 01:18:58.481273 (+ 95us) client_negotiation.cc:240] Negotiation successful
0812 01:18:58.481331 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":47,"threads_started":1}
I20260812 01:18:58.481419 26232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:18:58.477293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57620 (local address 127.24.0.130:44769)
0812 01:18:58.477452 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0812 01:18:58.477457 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 01:18:58.477469 (+ 12us) server_negotiation.cc:407] Connection header received
0812 01:18:58.477551 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:18:58.477554 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:18:58.477570 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:18:58.477655 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:18:58.477825 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:18:58.478805 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:18:58.480325 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:18:58.480979 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:18:58.481069 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:18:58.481133 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 01:18:58.481295 (+ 162us) server_negotiation.cc:299] Negotiation successful
0812 01:18:58.481337 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":94,"threads_started":1}
I20260812 01:18:58.481693 26222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:18:58.477024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.129:41267 (local address 127.0.0.1:50510)
0812 01:18:58.477488 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0812 01:18:58.477498 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 01:18:58.477566 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:18:58.477833 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:18:58.477834 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:18:58.477836 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:18:58.477880 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:18:58.477882 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:18:58.479141 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:18:58.479141 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:18:58.480441 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:18:58.480443 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:18:58.481479 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:18:58.481479 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:18:58.481535 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:18:58.481538 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 01:18:58.481588 (+ 50us) client_negotiation.cc:240] Negotiation successful
0812 01:18:58.481624 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":119,"threads_started":1}
I20260812 01:18:58.481711 26228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:18:58.477246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37532 (local address 127.24.0.131:38317)
0812 01:18:58.477426 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0812 01:18:58.477430 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 01:18:58.477756 (+ 326us) server_negotiation.cc:407] Connection header received
0812 01:18:58.477799 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:18:58.477801 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:18:58.477818 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:18:58.477908 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:18:58.478119 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:18:58.479156 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:18:58.480314 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:18:58.481108 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:18:58.481195 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:18:58.481251 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 01:18:58.481597 (+ 346us) server_negotiation.cc:299] Negotiation successful
0812 01:18:58.481636 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":79,"threads_started":1}
I20260812 01:18:58.481524 26223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:18:58.477034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.131:38317 (local address 127.0.0.1:37532)
0812 01:18:58.477646 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0812 01:18:58.477655 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 01:18:58.477702 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:18:58.477991 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:18:58.477993 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:18:58.477995 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:18:58.478039 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:18:58.478041 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:18:58.479275 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:18:58.479276 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:18:58.480230 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:18:58.480233 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:18:58.481290 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:18:58.481291 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:18:58.481354 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:18:58.481358 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 01:18:58.481406 (+ 48us) client_negotiation.cc:240] Negotiation successful
0812 01:18:58.481449 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":596,"mutex_wait_us":17,"thread_start_us":79,"threads_started":1}
I20260812 01:18:58.481992 26225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:18:58.477139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50510 (local address 127.24.0.129:41267)
0812 01:18:58.477322 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0812 01:18:58.477327 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 01:18:58.477577 (+ 250us) server_negotiation.cc:407] Connection header received
0812 01:18:58.477641 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:18:58.477644 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:18:58.477660 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:18:58.477742 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:18:58.477939 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:18:58.479035 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:18:58.480520 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:18:58.481328 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:18:58.481394 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:18:58.481446 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 01:18:58.481867 (+ 421us) server_negotiation.cc:299] Negotiation successful
0812 01:18:58.481913 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":92,"threads_started":1}
I20260812 01:18:58.482445 26090 tablet_service.cc:1467] Processing CreateTablet for tablet dbd6a1ad22a249bb99b81a590ebb14b1 (DEFAULT_TABLE table=test-table [id=54d19a5f049041f595fbd406307bed85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:18:58.482431 25885 tablet_service.cc:1467] Processing CreateTablet for tablet dbd6a1ad22a249bb99b81a590ebb14b1 (DEFAULT_TABLE table=test-table [id=54d19a5f049041f595fbd406307bed85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:18:58.482524 25677 tablet_service.cc:1467] Processing CreateTablet for tablet dbd6a1ad22a249bb99b81a590ebb14b1 (DEFAULT_TABLE table=test-table [id=54d19a5f049041f595fbd406307bed85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:18:58.482784 25677 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dbd6a1ad22a249bb99b81a590ebb14b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:18:58.482795 25885 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dbd6a1ad22a249bb99b81a590ebb14b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:18:58.482795 26090 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dbd6a1ad22a249bb99b81a590ebb14b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:18:58.485487 26233 tablet_bootstrap.cc:492] T dbd6a1ad22a249bb99b81a590ebb14b1 P b5bcf4602b864e5b9758131bedb6a75d: Bootstrap starting.
I20260812 01:18:58.486071 26237 tablet_bootstrap.cc:492] T dbd6a1ad22a249bb99b81a590ebb14b1 P b104dce7b86f4602a527379ea53ae301: Bootstrap starting.
I20260812 01:18:58.486121 26233 tablet_bootstrap.cc:654] T dbd6a1ad22a249bb99b81a590ebb14b1 P b5bcf4602b864e5b9758131bedb6a75d: Neither blocks nor log segments found. Creating new log.
I20260812 01:18:58.486774 26237 tablet_bootstrap.cc:654] T dbd6a1ad22a249bb99b81a590ebb14b1 P b104dce7b86f4602a527379ea53ae301: Neither blocks nor log segments found. Creating new log.
I20260812 01:18:58.486749 26233 log.cc:824] T dbd6a1ad22a249bb99b81a590ebb14b1 P b5bcf4602b864e5b9758131bedb6a75d: Log is configured to *not* fsync() on all Append() calls
I20260812 01:18:58.487169 26234 tablet_bootstrap.cc:492] T dbd6a1ad22a249bb99b81a590ebb14b1 P 22343a784b424372b7677f7f66581e51: Bootstrap starting.
I20260812 01:18:58.487412 26237 log.cc:824] T dbd6a1ad22a249bb99b81a590ebb14b1 P b104dce7b86f4602a527379ea53ae301: Log is configured to *not* fsync() on all Append() calls
I20260812 01:18:58.487604 26233 tablet_bootstrap.cc:492] T dbd6a1ad22a249bb99b81a590ebb14b1 P b5bcf4602b864e5b9758131bedb6a75d: No bootstrap required, opened a new log
I20260812 01:18:58.487695 26233 ts_tablet_manager.cc:1397] T dbd6a1ad22a249bb99b81a590ebb14b1 P b5bcf4602b864e5b9758131bedb6a75d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 01:18:58.487933 26234 tablet_bootstrap.cc:654] T dbd6a1ad22a249bb99b81a590ebb14b1 P 22343a784b424372b7677f7f66581e51: Neither blocks nor log segments found. Creating new log.
I20260812 01:18:58.488456 26237 tablet_bootstrap.cc:492] T dbd6a1ad22a249bb99b81a590ebb14b1 P b104dce7b86f4602a527379ea53ae301: No bootstrap required, opened a new log
I20260812 01:18:58.488536 26237 ts_tablet_manager.cc:1397] T dbd6a1ad22a249bb99b81a590ebb14b1 P b104dce7b86f4602a527379ea53ae301: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 01:18:58.488636 26234 log.cc:824] T dbd6a1ad22a249bb99b81a590ebb14b1 P 22343a784b424372b7677f7f66581e51: Log is configured to *not* fsync() on all Append() calls
I20260812 01:18:58.489400 26234 tablet_bootstrap.cc:492] T dbd6a1ad22a249bb99b81a590ebb14b1 P 22343a784b424372b7677f7f66581e51: No bootstrap required, opened a new log
I20260812 01:18:58.489475 26234 ts_tablet_manager.cc:1397] T dbd6a1ad22a249bb99b81a590ebb14b1 P 22343a784b424372b7677f7f66581e51: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260812 01:18:58.489688 26233 raft_consensus.cc:348] T dbd6a1ad22a249bb99b81a590ebb14b1 P b5bcf4602b864e5b9758131bedb6a75d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22343a784b424372b7677f7f66581e51" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 44769 } } peers { permanent_uuid: "b5bcf4602b864e5b9758131bedb6a75d" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 41267 } } peers { permanent_uuid: "b104dce7b86f4602a527379ea53ae301" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 38317 } }
I20260812 01:18:58.489847 26233 raft_consensus.cc:374] T dbd6a1ad22a249bb99b81a590ebb14b1 P b5bcf4602b864e5b9758131bedb6a75d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:18:58.489893 26233 raft_consensus.cc:729] T dbd6a1ad22a249bb99b81a590ebb14b1 P b5bcf4602b864e5b9758131bedb6a75d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bcf4602b864e5b9758131bedb6a75d, State: Initialized, Role: FOLLOWER
I20260812 01:18:58.490010 26233 consensus_queue.cc:260] T dbd6a1ad22a249bb99b81a590ebb14b1 P b5bcf4602b864e5b9758131bedb6a75d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22343a784b424372b7677f7f66581e51" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 44769 } } peers { permanent_uuid: "b5bcf4602b864e5b9758131bedb6a75d" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 41267 } } peers { permanent_uuid: "b104dce7b86f4602a527379ea53ae301" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 38317 } }
I20260812 01:18:58.490325 26233 ts_tablet_manager.cc:1428] T dbd6a1ad22a249bb99b81a590ebb14b1 P b5bcf4602b864e5b9758131bedb6a75d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 01:18:58.490335 25780 heartbeater.cc:503] Master 127.24.0.190:39711 was elected leader, sending a full tablet report...
I20260812 01:18:58.490739 26237 raft_consensus.cc:348] T dbd6a1ad22a249bb99b81a590ebb14b1 P b104dce7b86f4602a527379ea53ae301 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22343a784b424372b7677f7f66581e51" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 44769 } } peers { permanent_uuid: "b5bcf4602b864e5b9758131bedb6a75d" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 41267 } } peers { permanent_uuid: "b104dce7b86f4602a527379ea53ae301" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 38317 } }
I20260812 01:18:58.490922 26237 raft_consensus.cc:374] T dbd6a1ad22a249bb99b81a590ebb14b1 P b104dce7b86f4602a527379ea53ae301 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:18:58.490968 26237 raft_consensus.cc:729] T dbd6a1ad22a249bb99b81a590ebb14b1 P b104dce7b86f4602a527379ea53ae301 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b104dce7b86f4602a527379ea53ae301, State: Initialized, Role: FOLLOWER
I20260812 01:18:58.491072 26237 consensus_queue.cc:260] T dbd6a1ad22a249bb99b81a590ebb14b1 P b104dce7b86f4602a527379ea53ae301 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22343a784b424372b7677f7f66581e51" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 44769 } } peers { permanent_uuid: "b5bcf4602b864e5b9758131bedb6a75d" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 41267 } } peers { permanent_uuid: "b104dce7b86f4602a527379ea53ae301" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 38317 } }
I20260812 01:18:58.491361 26237 ts_tablet_manager.cc:1428] T dbd6a1ad22a249bb99b81a590ebb14b1 P b104dce7b86f4602a527379ea53ae301: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 01:18:58.491492 26207 heartbeater.cc:503] Master 127.24.0.190:39711 was elected leader, sending a full tablet report...
I20260812 01:18:58.491503 26234 raft_consensus.cc:348] T dbd6a1ad22a249bb99b81a590ebb14b1 P 22343a784b424372b7677f7f66581e51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22343a784b424372b7677f7f66581e51" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 44769 } } peers { permanent_uuid: "b5bcf4602b864e5b9758131bedb6a75d" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 41267 } } peers { permanent_uuid: "b104dce7b86f4602a527379ea53ae301" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 38317 } }
I20260812 01:18:58.491678 26234 raft_consensus.cc:374] T dbd6a1ad22a249bb99b81a590ebb14b1 P 22343a784b424372b7677f7f66581e51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:18:58.491734 26234 raft_consensus.cc:729] T dbd6a1ad22a249bb99b81a590ebb14b1 P 22343a784b424372b7677f7f66581e51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22343a784b424372b7677f7f66581e51, State: Initialized, Role: FOLLOWER
I20260812 01:18:58.491848 26234 consensus_queue.cc:260] T dbd6a1ad22a249bb99b81a590ebb14b1 P 22343a784b424372b7677f7f66581e51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22343a784b424372b7677f7f66581e51" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 44769 } } peers { permanent_uuid: "b5bcf4602b864e5b9758131bedb6a75d" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 41267 } } peers { permanent_uuid: "b104dce7b86f4602a527379ea53ae301" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 38317 } }
I20260812 01:18:58.492152 26234 ts_tablet_manager.cc:1428] T dbd6a1ad22a249bb99b81a590ebb14b1 P 22343a784b424372b7677f7f66581e51: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260812 01:18:58.492174 25981 heartbeater.cc:503] Master 127.24.0.190:39711 was elected leader, sending a full tablet report...
W20260812 01:18:58.533707 25985 tablet.cc:2369] T dbd6a1ad22a249bb99b81a590ebb14b1 P 22343a784b424372b7677f7f66581e51: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 01:18:58.612123 26243 raft_consensus.cc:482] T dbd6a1ad22a249bb99b81a590ebb14b1 P b104dce7b86f4602a527379ea53ae301 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 01:18:58.612262 26243 raft_consensus.cc:504] T dbd6a1ad22a249bb99b81a590ebb14b1 P b104dce7b86f4602a527379ea53ae301 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22343a784b424372b7677f7f66581e51" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 44769 } } peers { permanent_uuid: "b5bcf4602b864e5b9758131bedb6a75d" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 41267 } } peers { permanent_uuid: "b104dce7b86f4602a527379ea53ae301" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 38317 } }
I20260812 01:18:58.612690 26243 leader_election.cc:288] T dbd6a1ad22a249bb99b81a590ebb14b1 P b104dce7b86f4602a527379ea53ae301 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 22343a784b424372b7677f7f66581e51 (127.24.0.130:44769), b5bcf4602b864e5b9758131bedb6a75d (127.24.0.129:41267)
I20260812 01:18:58.615922 26225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:18:58.613081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.131:50865 (local address 127.24.0.129:41267)
0812 01:18:58.613135 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0812 01:18:58.613137 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:18:58.613149 (+ 12us) server_negotiation.cc:407] Connection header received
0812 01:18:58.613209 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:18:58.613210 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:18:58.613226 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:18:58.613294 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:18:58.613452 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:18:58.614131 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:18:58.615117 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:18:58.615588 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:18:58.615651 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:18:58.615677 (+ 26us) server_negotiation.cc:1035] Waiting for connection context
0812 01:18:58.615820 (+ 143us) server_negotiation.cc:299] Negotiation successful
0812 01:18:58.615852 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 01:18:58.615943 26209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:18:58.613015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.129:41267 (local address 127.24.0.131:50865)
0812 01:18:58.613061 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0812 01:18:58.613074 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 01:18:58.613131 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:18:58.613310 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:18:58.613312 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:18:58.613314 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:18:58.613378 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:18:58.613380 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:18:58.614228 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:18:58.614229 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:18:58.615019 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:18:58.615021 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:18:58.615669 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:18:58.615670 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:18:58.615748 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:18:58.615752 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 01:18:58.615813 (+ 61us) client_negotiation.cc:240] Negotiation successful
0812 01:18:58.615851 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24,"mutex_wait_us":1}
I20260812 01:18:58.616307 26232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:18:58.613008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.131:42037 (local address 127.24.0.130:44769)
0812 01:18:58.613063 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0812 01:18:58.613065 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:18:58.613518 (+ 453us) server_negotiation.cc:407] Connection header received
0812 01:18:58.613579 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:18:58.613580 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:18:58.613591 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:18:58.613639 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:18:58.613804 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:18:58.614303 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:18:58.615375 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:18:58.615871 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:18:58.615934 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:18:58.615941 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0812 01:18:58.616187 (+ 246us) server_negotiation.cc:299] Negotiation successful
0812 01:18:58.616229 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260812 01:18:58.616361 25706 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbd6a1ad22a249bb99b81a590ebb14b1" candidate_uuid: "b104dce7b86f4602a527379ea53ae301" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bcf4602b864e5b9758131bedb6a75d" is_pre_election: true
I20260812 01:18:58.616556 26245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:18:58.612981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.130:44769 (local address 127.24.0.131:42037)
0812 01:18:58.613431 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0812 01:18:58.613443 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 01:18:58.613516 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:18:58.613664 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:18:58.613665 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:18:58.613667 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:18:58.613740 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:18:58.613742 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:18:58.614405 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:18:58.614406 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:18:58.615284 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:18:58.615287 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:18:58.615964 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:18:58.615965 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:18:58.616078 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:18:58.616083 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 01:18:58.616397 (+ 314us) client_negotiation.cc:240] Negotiation successful
0812 01:18:58.616441 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":385,"mutex_wait_us":175,"thread_start_us":155,"threads_started":1}
I20260812 01:18:58.616549 25706 raft_consensus.cc:2454] T dbd6a1ad22a249bb99b81a590ebb14b1 P b5bcf4602b864e5b9758131bedb6a75d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b104dce7b86f4602a527379ea53ae301 in term 0.
I20260812 01:18:58.616832 26016 leader_election.cc:302] T dbd6a1ad22a249bb99b81a590ebb14b1 P b104dce7b86f4602a527379ea53ae301 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b104dce7b86f4602a527379ea53ae301, b5bcf4602b864e5b9758131bedb6a75d; no voters:
I20260812 01:18:58.616964 26243 raft_consensus.cc:2781] T dbd6a1ad22a249bb99b81a590ebb14b1 P b104dce7b86f4602a527379ea53ae301 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 01:18:58.617054 26243 raft_consensus.cc:482] T dbd6a1ad22a249bb99b81a590ebb14b1 P b104dce7b86f4602a527379ea53ae301 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 01:18:58.617081 26243 raft_consensus.cc:3037] T dbd6a1ad22a249bb99b81a590ebb14b1 P b104dce7b86f4602a527379ea53ae301 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:18:58.617164 25918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbd6a1ad22a249bb99b81a590ebb14b1" candidate_uuid: "b104dce7b86f4602a527379ea53ae301" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22343a784b424372b7677f7f66581e51" is_pre_election: true
I20260812 01:18:58.617334 25918 raft_consensus.cc:2454] T dbd6a1ad22a249bb99b81a590ebb14b1 P 22343a784b424372b7677f7f66581e51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b104dce7b86f4602a527379ea53ae301 in term 0.
I20260812 01:18:58.617955 26243 raft_consensus.cc:504] T dbd6a1ad22a249bb99b81a590ebb14b1 P b104dce7b86f4602a527379ea53ae301 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22343a784b424372b7677f7f66581e51" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 44769 } } peers { permanent_uuid: "b5bcf4602b864e5b9758131bedb6a75d" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 41267 } } peers { permanent_uuid: "b104dce7b86f4602a527379ea53ae301" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 38317 } }
I20260812 01:18:58.618098 26243 leader_election.cc:288] T dbd6a1ad22a249bb99b81a590ebb14b1 P b104dce7b86f4602a527379ea53ae301 [CANDIDATE]: Term 1 election: Requested vote from peers 22343a784b424372b7677f7f66581e51 (127.24.0.130:44769), b5bcf4602b864e5b9758131bedb6a75d (127.24.0.129:41267)
I20260812 01:18:58.618232 25918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbd6a1ad22a249bb99b81a590ebb14b1" candidate_uuid: "b104dce7b86f4602a527379ea53ae301" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22343a784b424372b7677f7f66581e51"
I20260812 01:18:58.618242 25706 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbd6a1ad22a249bb99b81a590ebb14b1" candidate_uuid: "b104dce7b86f4602a527379ea53ae301" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bcf4602b864e5b9758131bedb6a75d"
I20260812 01:18:58.618319 25706 raft_consensus.cc:3037] T dbd6a1ad22a249bb99b81a590ebb14b1 P b5bcf4602b864e5b9758131bedb6a75d [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:18:58.618323 25918 raft_consensus.cc:3037] T dbd6a1ad22a249bb99b81a590ebb14b1 P 22343a784b424372b7677f7f66581e51 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:18:58.619057 25706 raft_consensus.cc:2454] T dbd6a1ad22a249bb99b81a590ebb14b1 P b5bcf4602b864e5b9758131bedb6a75d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b104dce7b86f4602a527379ea53ae301 in term 1.
I20260812 01:18:58.619086 25918 raft_consensus.cc:2454] T dbd6a1ad22a249bb99b81a590ebb14b1 P 22343a784b424372b7677f7f66581e51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b104dce7b86f4602a527379ea53ae301 in term 1.
I20260812 01:18:58.619261 26014 leader_election.cc:302] T dbd6a1ad22a249bb99b81a590ebb14b1 P b104dce7b86f4602a527379ea53ae301 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22343a784b424372b7677f7f66581e51, b104dce7b86f4602a527379ea53ae301; no voters:
I20260812 01:18:58.619396 26243 raft_consensus.cc:2781] T dbd6a1ad22a249bb99b81a590ebb14b1 P b104dce7b86f4602a527379ea53ae301 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:18:58.619551 26243 raft_consensus.cc:686] T dbd6a1ad22a249bb99b81a590ebb14b1 P b104dce7b86f4602a527379ea53ae301 [term 1 LEADER]: Becoming Leader. State: Replica: b104dce7b86f4602a527379ea53ae301, State: Running, Role: LEADER
I20260812 01:18:58.619659 26243 consensus_queue.cc:237] T dbd6a1ad22a249bb99b81a590ebb14b1 P b104dce7b86f4602a527379ea53ae301 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22343a784b424372b7677f7f66581e51" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 44769 } } peers { permanent_uuid: "b5bcf4602b864e5b9758131bedb6a75d" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 41267 } } peers { permanent_uuid: "b104dce7b86f4602a527379ea53ae301" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 38317 } }
I20260812 01:18:58.620592 25489 catalog_manager.cc:5495] T dbd6a1ad22a249bb99b81a590ebb14b1 P b104dce7b86f4602a527379ea53ae301 reported cstate change: term changed from 0 to 1, leader changed from <none> to b104dce7b86f4602a527379ea53ae301 (127.24.0.131). New cstate: current_term: 1 leader_uuid: "b104dce7b86f4602a527379ea53ae301" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22343a784b424372b7677f7f66581e51" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 44769 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5bcf4602b864e5b9758131bedb6a75d" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 41267 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b104dce7b86f4602a527379ea53ae301" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 38317 } health_report { overall_health: HEALTHY } } }
W20260812 01:18:58.621064 25781 tablet.cc:2369] T dbd6a1ad22a249bb99b81a590ebb14b1 P b5bcf4602b864e5b9758131bedb6a75d: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 12 01:18:58 dist-test-slave-t3q3 krb5kdc[25421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497537, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.0.129@KRBTEST.COM
I20260812 01:18:58.632745 26225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:18:58.625836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50520 (local address 127.24.0.129:41267)
0812 01:18:58.625915 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0812 01:18:58.625918 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:18:58.625988 (+ 70us) server_negotiation.cc:407] Connection header received
0812 01:18:58.626038 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:18:58.626039 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:18:58.626054 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:18:58.626095 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:18:58.626647 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:18:58.627116 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:18:58.627788 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:18:58.627979 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:18:58.628039 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:18:58.630293 (+ 2254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:18:58.630301 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:18:58.630301 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:18:58.630307 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:18:58.631701 (+ 1394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:18:58.632003 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:18:58.632006 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:18:58.632006 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:18:58.632131 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:18:58.632304 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:18:58.632305 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:18:58.632305 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:18:58.632435 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:18:58.632536 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0812 01:18:58.632591 (+ 55us) server_negotiation.cc:299] Negotiation successful
0812 01:18:58.632651 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
Aug 12 01:18:58 dist-test-slave-t3q3 krb5kdc[25421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497538, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 01:18:58 dist-test-slave-t3q3 krb5kdc[25421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497538, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.24.0.129@KRBTEST.COM
I20260812 01:18:58.657559 26225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:18:58.650442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50530 (local address 127.24.0.129:41267)
0812 01:18:58.650511 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0812 01:18:58.650513 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:18:58.650596 (+ 83us) server_negotiation.cc:407] Connection header received
0812 01:18:58.650679 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:18:58.650680 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:18:58.650690 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:18:58.650763 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:18:58.651304 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:18:58.651816 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:18:58.652608 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:18:58.652822 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:18:58.652896 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:18:58.655078 (+ 2182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:18:58.655080 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:18:58.655080 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:18:58.655086 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:18:58.656455 (+ 1369us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:18:58.656774 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:18:58.656775 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:18:58.656775 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:18:58.656828 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:18:58.656989 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:18:58.656990 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:18:58.656990 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:18:58.657077 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:18:58.657150 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 01:18:58.657406 (+ 256us) server_negotiation.cc:299] Negotiation successful
0812 01:18:58.657489 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
Aug 12 01:18:58 dist-test-slave-t3q3 krb5kdc[25421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497538, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 01:18:58 dist-test-slave-t3q3 krb5kdc[25421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497538, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.0.129@KRBTEST.COM
I20260812 01:18:58.681679 26225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:18:58.675280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50538 (local address 127.24.0.129:41267)
0812 01:18:58.675356 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0812 01:18:58.675358 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:18:58.675407 (+ 49us) server_negotiation.cc:407] Connection header received
0812 01:18:58.675453 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:18:58.675454 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:18:58.675464 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:18:58.675511 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:18:58.676029 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:18:58.676540 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:18:58.677245 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:18:58.677420 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:18:58.677470 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:18:58.679538 (+ 2068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:18:58.679539 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:18:58.679539 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:18:58.679546 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:18:58.680821 (+ 1275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:18:58.681120 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:18:58.681121 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:18:58.681121 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:18:58.681170 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:18:58.681326 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:18:58.681326 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:18:58.681327 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:18:58.681412 (+ 85us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:18:58.681480 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 01:18:58.681562 (+ 82us) server_negotiation.cc:299] Negotiation successful
0812 01:18:58.681617 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
W20260812 01:18:58.682137 25645 tablet_service.cc:2908] Rejecting scan request for tablet dbd6a1ad22a249bb99b81a590ebb14b1: Uninitialized: clean time has not yet been initialized
I20260812 01:18:58.682976 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 25590
I20260812 01:18:58.697031 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 25788
W20260812 01:18:58.701651 26208 tablet.cc:2369] T dbd6a1ad22a249bb99b81a590ebb14b1 P b104dce7b86f4602a527379ea53ae301: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 01:18:58.725976 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 25990
I20260812 01:18:58.741266 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 25441
2026-08-12T01:18:58Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2349 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 01:18:58 dist-test-slave-t3q3 krb5kdc[26269](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 01:18:58 dist-test-slave-t3q3 krb5kdc[26269](info): set up 2 sockets
Aug 12 01:18:58 dist-test-slave-t3q3 krb5kdc[26269](info): commencing operation
krb5kdc: starting...
W20260812 01:19:00.020576 24578 mini_kdc.cc:121] Time spent starting KDC: real 1.247s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/krb5kdc/test-user.keytab.
Aug 12 01:19:00 dist-test-slave-t3q3 krb5kdc[26269](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497540, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T01:19:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:19:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.0.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.0.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.0.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab.
Entry for principal kudu/127.24.0.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab.
I20260812 01:19:00.140801 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.190:42265
--webserver_interface=127.24.0.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab
--principal=kudu/127.24.0.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:39799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.0.190:42265
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:19:00.223657 26288 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:19:00.223896 26288 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:19:00.223934 26288 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:19:00.225129 26288 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:19:00.225203 26288 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260812 01:19:00.225225 26288 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:19:00.225257 26288 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:19:00.225282 26288 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:19:00.225304 26288 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:19:00.226752 26288 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:39799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.0.190:42265
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab
--principal=kudu/127.24.0.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.190:42265
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.0.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26288
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:19:00.227023 26288 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:19:00.227253 26288 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:19:00.232028 26296 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:00.232041 26298 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:00.232101 26293 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:19:00.232295 26288 server_base.cc:1034] running on GCE node
I20260812 01:19:00.232528 26288 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:19:00.232765 26288 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:19:00.233906 26288 hybrid_clock.cc:648] HybridClock initialized: now 1786497540233885 us; error 36 us; skew 500 ppm
Aug 12 01:19:00 dist-test-slave-t3q3 krb5kdc[26269](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497540, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:19:00.236397 26288 init.cc:377] Logged in from keytab as kudu/127.24.0.190@KRBTEST.COM (short username kudu)
I20260812 01:19:00.236862 26288 webserver.cc:460] Webserver started at http://127.24.0.190:36507/ using document root <none> and password file <none>
I20260812 01:19:00.237083 26288 fs_manager.cc:359] Metadata directory not provided
I20260812 01:19:00.237147 26288 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:19:00.237299 26288 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:19:00.238348 26288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/minicluster-data/master-0/data/instance:
uuid: "dec86a9c3f9845a383f2cc0f2a31b340"
format_stamp: "Formatted at 2026-08-12 01:19:00 on dist-test-slave-t3q3"
server_key: "2b8e200273e5f1ac2283eff8d2fefdf2"
server_key_iv: "7bee26171cf627540de4ca40fa568445"
server_key_version: "encryptionkey@0"
I20260812 01:19:00.238704 26288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/minicluster-data/master-0/wal/instance:
uuid: "dec86a9c3f9845a383f2cc0f2a31b340"
format_stamp: "Formatted at 2026-08-12 01:19:00 on dist-test-slave-t3q3"
server_key: "2b8e200273e5f1ac2283eff8d2fefdf2"
server_key_iv: "7bee26171cf627540de4ca40fa568445"
server_key_version: "encryptionkey@0"
I20260812 01:19:00.240146 26288 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260812 01:19:00.240875 26307 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:00.241044 26288 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 01:19:00.241142 26288 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/minicluster-data/master-0/wal
uuid: "dec86a9c3f9845a383f2cc0f2a31b340"
format_stamp: "Formatted at 2026-08-12 01:19:00 on dist-test-slave-t3q3"
server_key: "2b8e200273e5f1ac2283eff8d2fefdf2"
server_key_iv: "7bee26171cf627540de4ca40fa568445"
server_key_version: "encryptionkey@0"
I20260812 01:19:00.241233 26288 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:19:00.254382 26288 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:19:00.256283 26288 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:19:00.256439 26288 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:19:00.261482 26288 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.190:42265
I20260812 01:19:00.261538 26406 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.190:42265 every 8 connection(s)
I20260812 01:19:00.261847 26288 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
I20260812 01:19:00.262480 26408 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:19:00.264878 26408 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dec86a9c3f9845a383f2cc0f2a31b340: Bootstrap starting.
I20260812 01:19:00.265522 26408 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dec86a9c3f9845a383f2cc0f2a31b340: Neither blocks nor log segments found. Creating new log.
I20260812 01:19:00.265870 26408 log.cc:824] T 00000000000000000000000000000000 P dec86a9c3f9845a383f2cc0f2a31b340: Log is configured to *not* fsync() on all Append() calls
I20260812 01:19:00.265988 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 26288
I20260812 01:19:00.266086 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/minicluster-data/master-0/wal/instance
I20260812 01:19:00.266287 24578 external_mini_cluster.cc:1428] Setting key 01a40a2859cfdb8608a9c5d2f8d4d7d8
I20260812 01:19:00.266711 26408 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dec86a9c3f9845a383f2cc0f2a31b340: No bootstrap required, opened a new log
I20260812 01:19:00.269148 26408 raft_consensus.cc:348] T 00000000000000000000000000000000 P dec86a9c3f9845a383f2cc0f2a31b340 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dec86a9c3f9845a383f2cc0f2a31b340" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 42265 } }
I20260812 01:19:00.269310 26408 raft_consensus.cc:374] T 00000000000000000000000000000000 P dec86a9c3f9845a383f2cc0f2a31b340 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:19:00.269358 26408 raft_consensus.cc:729] T 00000000000000000000000000000000 P dec86a9c3f9845a383f2cc0f2a31b340 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dec86a9c3f9845a383f2cc0f2a31b340, State: Initialized, Role: FOLLOWER
I20260812 01:19:00.269523 26408 consensus_queue.cc:260] T 00000000000000000000000000000000 P dec86a9c3f9845a383f2cc0f2a31b340 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dec86a9c3f9845a383f2cc0f2a31b340" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 42265 } }
I20260812 01:19:00.269616 26408 raft_consensus.cc:388] T 00000000000000000000000000000000 P dec86a9c3f9845a383f2cc0f2a31b340 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:19:00.269667 26408 raft_consensus.cc:482] T 00000000000000000000000000000000 P dec86a9c3f9845a383f2cc0f2a31b340 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:19:00.269717 26408 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dec86a9c3f9845a383f2cc0f2a31b340 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:19:00.270395 26408 raft_consensus.cc:504] T 00000000000000000000000000000000 P dec86a9c3f9845a383f2cc0f2a31b340 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dec86a9c3f9845a383f2cc0f2a31b340" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 42265 } }
I20260812 01:19:00.270510 26408 leader_election.cc:302] T 00000000000000000000000000000000 P dec86a9c3f9845a383f2cc0f2a31b340 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dec86a9c3f9845a383f2cc0f2a31b340; no voters:
I20260812 01:19:00.270691 26408 leader_election.cc:288] T 00000000000000000000000000000000 P dec86a9c3f9845a383f2cc0f2a31b340 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 01:19:00.270746 26418 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dec86a9c3f9845a383f2cc0f2a31b340 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:19:00.270936 26408 sys_catalog.cc:564] T 00000000000000000000000000000000 P dec86a9c3f9845a383f2cc0f2a31b340 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:19:00.270936 26418 raft_consensus.cc:686] T 00000000000000000000000000000000 P dec86a9c3f9845a383f2cc0f2a31b340 [term 1 LEADER]: Becoming Leader. State: Replica: dec86a9c3f9845a383f2cc0f2a31b340, State: Running, Role: LEADER
I20260812 01:19:00.271072 26418 consensus_queue.cc:237] T 00000000000000000000000000000000 P dec86a9c3f9845a383f2cc0f2a31b340 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dec86a9c3f9845a383f2cc0f2a31b340" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 42265 } }
I20260812 01:19:00.271405 26418 sys_catalog.cc:455] T 00000000000000000000000000000000 P dec86a9c3f9845a383f2cc0f2a31b340 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dec86a9c3f9845a383f2cc0f2a31b340. Latest consensus state: current_term: 1 leader_uuid: "dec86a9c3f9845a383f2cc0f2a31b340" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dec86a9c3f9845a383f2cc0f2a31b340" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 42265 } } }
I20260812 01:19:00.271498 26418 sys_catalog.cc:458] T 00000000000000000000000000000000 P dec86a9c3f9845a383f2cc0f2a31b340 [sys.catalog]: This master's current role is: LEADER
I20260812 01:19:00.271415 26423 sys_catalog.cc:455] T 00000000000000000000000000000000 P dec86a9c3f9845a383f2cc0f2a31b340 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dec86a9c3f9845a383f2cc0f2a31b340" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dec86a9c3f9845a383f2cc0f2a31b340" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 42265 } } }
I20260812 01:19:00.271637 26423 sys_catalog.cc:458] T 00000000000000000000000000000000 P dec86a9c3f9845a383f2cc0f2a31b340 [sys.catalog]: This master's current role is: LEADER
I20260812 01:19:00.271718 26432 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:19:00.272838 26432 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 12 01:19:00 dist-test-slave-t3q3 krb5kdc[26269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497540, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
I20260812 01:19:00.274873 26432 catalog_manager.cc:1347] Generated new cluster ID: c04b910b129148039c21f3eaf2fa1a85
I20260812 01:19:00.274927 26432 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 01:19:00.279549 26415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:00.266896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59870 (local address 127.24.0.190:42265)
0812 01:19:00.267067 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:00.267075 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:00.267102 (+ 27us) server_negotiation.cc:407] Connection header received
0812 01:19:00.267522 (+ 420us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:00.267529 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:00.267575 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:00.267712 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:00.268556 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:00.269930 (+ 1374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:00.271013 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:00.271696 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:00.271815 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:00.275049 (+ 3234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:00.275053 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:00.275054 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:00.275066 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:00.277207 (+ 2141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:00.277629 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:00.277633 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:00.277633 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:00.277754 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:00.277973 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:00.277974 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:00.277974 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:00.278287 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:00.278457 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:00.278936 (+ 479us) server_negotiation.cc:299] Negotiation successful
0812 01:19:00.279084 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":84,"threads_started":1}
I20260812 01:19:00.283913 26432 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:19:00.284472 26432 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:19:00.290529 26432 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dec86a9c3f9845a383f2cc0f2a31b340: Generated new TSK 0
I20260812 01:19:00.290725 26432 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.0.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.0.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.0.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.129.keytab.
Entry for principal kudu/127.24.0.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.129.keytab.
I20260812 01:19:00.319186 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.0.129:0
--local_ip_for_outbound_sockets=127.24.0.129
--webserver_interface=127.24.0.129
--webserver_port=0
--tserver_master_addrs=127.24.0.190:42265
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.129.keytab
--principal=kudu/127.24.0.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:39799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:19:00.405850 26450 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:19:00.406090 26450 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:19:00.406132 26450 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:19:00.407285 26450 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:19:00.407347 26450 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:19:00.407383 26450 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 01:19:00.407433 26450 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.0.129
I20260812 01:19:00.408963 26450 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:39799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.129.keytab
--principal=kudu/127.24.0.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.0.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.0.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.0.190:42265
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26450
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.0.129
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:19:00.409225 26450 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:19:00.409458 26450 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:19:00.414705 26460 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:00.414654 26464 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:19:00.414682 26450 server_base.cc:1034] running on GCE node
W20260812 01:19:00.414660 26459 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:19:00.415176 26450 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:19:00.415421 26450 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:19:00.416575 26450 hybrid_clock.cc:648] HybridClock initialized: now 1786497540416564 us; error 40 us; skew 500 ppm
Aug 12 01:19:00 dist-test-slave-t3q3 krb5kdc[26269](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497540, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:19:00.419039 26450 init.cc:377] Logged in from keytab as kudu/127.24.0.129@KRBTEST.COM (short username kudu)
I20260812 01:19:00.419471 26450 webserver.cc:460] Webserver started at http://127.24.0.129:36273/ using document root <none> and password file <none>
I20260812 01:19:00.419692 26450 fs_manager.cc:359] Metadata directory not provided
I20260812 01:19:00.419757 26450 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:19:00.419904 26450 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:19:00.421027 26450 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/minicluster-data/ts-0/data/instance:
uuid: "6944457d5a3046a2b3623fc868ee3aa1"
format_stamp: "Formatted at 2026-08-12 01:19:00 on dist-test-slave-t3q3"
server_key: "70af9a246064f50cf9f106cf28e2cbc2"
server_key_iv: "df30c125c90ba586c2836f579605af3c"
server_key_version: "encryptionkey@0"
I20260812 01:19:00.421389 26450 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/minicluster-data/ts-0/wal/instance:
uuid: "6944457d5a3046a2b3623fc868ee3aa1"
format_stamp: "Formatted at 2026-08-12 01:19:00 on dist-test-slave-t3q3"
server_key: "70af9a246064f50cf9f106cf28e2cbc2"
server_key_iv: "df30c125c90ba586c2836f579605af3c"
server_key_version: "encryptionkey@0"
I20260812 01:19:00.422811 26450 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 01:19:00.423516 26472 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:00.423691 26450 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 01:19:00.423792 26450 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/minicluster-data/ts-0/wal
uuid: "6944457d5a3046a2b3623fc868ee3aa1"
format_stamp: "Formatted at 2026-08-12 01:19:00 on dist-test-slave-t3q3"
server_key: "70af9a246064f50cf9f106cf28e2cbc2"
server_key_iv: "df30c125c90ba586c2836f579605af3c"
server_key_version: "encryptionkey@0"
I20260812 01:19:00.423866 26450 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:19:00.458668 26450 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:19:00.460688 26450 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:19:00.460855 26450 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:19:00.461404 26450 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:19:00.461463 26450 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:00.461506 26450 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:19:00.461547 26450 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:00.469000 26450 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.129:38473
I20260812 01:19:00.469063 26654 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.129:38473 every 8 connection(s)
Aug 12 01:19:00 dist-test-slave-t3q3 krb5kdc[26269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497540, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.129@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
I20260812 01:19:00.469385 26450 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/minicluster-data/ts-0/data/info.pb
I20260812 01:19:00.472440 26415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:00.462084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.129:44815 (local address 127.24.0.190:42265)
0812 01:19:00.462137 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:00.462139 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:00.462375 (+ 236us) server_negotiation.cc:407] Connection header received
0812 01:19:00.462955 (+ 580us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:00.462956 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:00.462973 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:00.463042 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:00.464003 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:00.464870 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:00.465978 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:00.466344 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:00.466421 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:00.469871 (+ 3450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:00.469872 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:00.469872 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:00.469880 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:00.471189 (+ 1309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:00.471542 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:00.471542 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:00.471543 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:00.471596 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:00.471799 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:00.471800 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:00.471800 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:00.471899 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:00.471950 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:00.472290 (+ 340us) server_negotiation.cc:299] Negotiation successful
0812 01:19:00.472373 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 01:19:00.472569 26491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:00.462091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:42265 (local address 127.24.0.129:44815)
0812 01:19:00.462253 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:00.462288 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:00.462797 (+ 509us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:00.463155 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:00.463159 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:00.463561 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:00.463884 (+ 323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:00.463892 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:00.464975 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:00.464976 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:00.465889 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:00.465892 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:00.466462 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:00.466463 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:00.466633 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:00.467940 (+ 1307us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:00.467951 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:00.469785 (+ 1834us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:19:00.471260 (+ 1475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:00.471266 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:00.471268 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:00.471479 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:00.471650 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:00.471650 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:00.471651 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:00.471748 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:00.471950 (+ 202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:19:00.471954 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:19:00.472163 (+ 209us) client_negotiation.cc:769] Sending connection context
0812 01:19:00.472255 (+ 92us) client_negotiation.cc:240] Negotiation successful
0812 01:19:00.472374 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":84,"threads_started":1}
I20260812 01:19:00.473083 26655 heartbeater.cc:347] Connected to a master server at 127.24.0.190:42265
I20260812 01:19:00.473177 26655 heartbeater.cc:464] Registering TS with master...
I20260812 01:19:00.473367 26655 heartbeater.cc:511] Master 127.24.0.190:42265 requested a full tablet report, sending...
I20260812 01:19:00.473852 26333 ts_manager.cc:194] Registered new tserver with Master: 6944457d5a3046a2b3623fc868ee3aa1 (127.24.0.129:38473)
I20260812 01:19:00.474524 26333 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.0.129@KRBTEST.COM'} at 127.24.0.129:44815
I20260812 01:19:00.475718 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 26450
I20260812 01:19:00.475819 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/minicluster-data/ts-0/wal/instance
I20260812 01:19:00.475993 24578 external_mini_cluster.cc:1428] Setting key 5a85b00e4a4edf26d3db2ce502c8e1e8
I20260812 01:19:00.480404 26667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:00.474220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:42265 (local address 127.24.0.129:47355)
0812 01:19:00.474382 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:00.474390 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:00.474439 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:00.474562 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:00.474563 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:00.474774 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:00.474871 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:00.474873 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:00.475531 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:00.475532 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:00.476364 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:00.476367 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:00.476806 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:00.476807 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:00.476866 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:00.477262 (+ 396us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:00.477266 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:00.477517 (+ 251us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:19:00.479083 (+ 1566us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:00.479084 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:00.479084 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:00.479297 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:00.479576 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:00.479577 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:00.479577 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:00.479675 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:00.480125 (+ 450us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:19:00.480126 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:19:00.480178 (+ 52us) client_negotiation.cc:769] Sending connection context
0812 01:19:00.480253 (+ 75us) client_negotiation.cc:240] Negotiation successful
0812 01:19:00.480330 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":65,"threads_started":1}
I20260812 01:19:00.480494 26415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:00.474274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.129:47355 (local address 127.24.0.190:42265)
0812 01:19:00.474324 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:00.474325 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:00.474445 (+ 120us) server_negotiation.cc:407] Connection header received
0812 01:19:00.474491 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:00.474491 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:00.474502 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:00.474561 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:00.474951 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:00.475434 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:00.476452 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:00.476728 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:00.476808 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:00.477597 (+ 789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:00.477598 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:00.477599 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:00.477606 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:00.479000 (+ 1394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:00.479426 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:00.479428 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:00.479428 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:00.479499 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:00.479819 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:00.479821 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:00.479821 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:00.480012 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:00.480122 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:00.480288 (+ 166us) server_negotiation.cc:299] Negotiation successful
0812 01:19:00.480376 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
WARNING: no policy specified for kudu/127.24.0.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.0.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.0.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.130.keytab.
Entry for principal kudu/127.24.0.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.130.keytab.
I20260812 01:19:00.502971 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.0.130:0
--local_ip_for_outbound_sockets=127.24.0.130
--webserver_interface=127.24.0.130
--webserver_port=0
--tserver_master_addrs=127.24.0.190:42265
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.130.keytab
--principal=kudu/127.24.0.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:39799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:19:00.586680 26672 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:19:00.586933 26672 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:19:00.586974 26672 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:19:00.588151 26672 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:19:00.588228 26672 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:19:00.588263 26672 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 01:19:00.588301 26672 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.0.130
I20260812 01:19:00.589766 26672 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:39799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.130.keytab
--principal=kudu/127.24.0.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.0.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.0.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.0.190:42265
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26672
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.0.130
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:19:00.590032 26672 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:19:00.590266 26672 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:19:00.595577 26683 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:00.595609 26686 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:19:00.595685 26672 server_base.cc:1034] running on GCE node
W20260812 01:19:00.595639 26688 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:19:00.595939 26672 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:19:00.596292 26672 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:19:00.597471 26672 hybrid_clock.cc:648] HybridClock initialized: now 1786497540597460 us; error 38 us; skew 500 ppm
Aug 12 01:19:00 dist-test-slave-t3q3 krb5kdc[26269](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497540, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:19:00.600143 26672 init.cc:377] Logged in from keytab as kudu/127.24.0.130@KRBTEST.COM (short username kudu)
I20260812 01:19:00.600646 26672 webserver.cc:460] Webserver started at http://127.24.0.130:42669/ using document root <none> and password file <none>
I20260812 01:19:00.600878 26672 fs_manager.cc:359] Metadata directory not provided
I20260812 01:19:00.600941 26672 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:19:00.601097 26672 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:19:00.602190 26672 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/minicluster-data/ts-1/data/instance:
uuid: "4ecd5f366111458aa0761a3ed177ae2e"
format_stamp: "Formatted at 2026-08-12 01:19:00 on dist-test-slave-t3q3"
server_key: "c33fbe02eb83970770b59bdd578f2ff2"
server_key_iv: "189475d4c38193e790c2bf57cf207926"
server_key_version: "encryptionkey@0"
I20260812 01:19:00.602567 26672 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/minicluster-data/ts-1/wal/instance:
uuid: "4ecd5f366111458aa0761a3ed177ae2e"
format_stamp: "Formatted at 2026-08-12 01:19:00 on dist-test-slave-t3q3"
server_key: "c33fbe02eb83970770b59bdd578f2ff2"
server_key_iv: "189475d4c38193e790c2bf57cf207926"
server_key_version: "encryptionkey@0"
I20260812 01:19:00.603992 26672 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 01:19:00.604780 26700 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:00.604959 26672 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 01:19:00.605062 26672 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/minicluster-data/ts-1/wal
uuid: "4ecd5f366111458aa0761a3ed177ae2e"
format_stamp: "Formatted at 2026-08-12 01:19:00 on dist-test-slave-t3q3"
server_key: "c33fbe02eb83970770b59bdd578f2ff2"
server_key_iv: "189475d4c38193e790c2bf57cf207926"
server_key_version: "encryptionkey@0"
I20260812 01:19:00.605145 26672 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:19:00.619897 26672 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:19:00.622269 26672 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:19:00.622435 26672 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:19:00.623073 26672 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:19:00.623132 26672 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:00.623188 26672 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:19:00.623217 26672 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 01:19:00 dist-test-slave-t3q3 krb5kdc[26269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497540, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.130@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
I20260812 01:19:00.631281 26672 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.130:38587
I20260812 01:19:00.631362 26904 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.130:38587 every 8 connection(s)
I20260812 01:19:00.631659 26672 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/minicluster-data/ts-1/data/info.pb
I20260812 01:19:00.632774 26415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:00.623464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.130:42251 (local address 127.24.0.190:42265)
0812 01:19:00.623523 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:00.623526 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:00.623866 (+ 340us) server_negotiation.cc:407] Connection header received
0812 01:19:00.624665 (+ 799us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:00.624666 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:00.624679 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:00.624716 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:00.625319 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:00.625817 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:00.626960 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:00.627154 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:00.627257 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:00.629874 (+ 2617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:00.629876 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:00.629877 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:00.629884 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:00.631449 (+ 1565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:00.631825 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:00.631827 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:00.631827 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:00.631892 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:00.632216 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:00.632217 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:00.632218 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:00.632333 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:00.632387 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:00.632646 (+ 259us) server_negotiation.cc:299] Negotiation successful
0812 01:19:00.632713 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 01:19:00.632958 26712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:00.623431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:42265 (local address 127.24.0.130:42251)
0812 01:19:00.623733 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:00.623776 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:00.624563 (+ 787us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:00.624813 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:00.624816 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:00.625043 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:00.625223 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:00.625227 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:00.625914 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:00.625915 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:00.626872 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:00.626875 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:00.627266 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:00.627267 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:00.627415 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:00.627952 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:00.627965 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:00.629778 (+ 1813us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:19:00.631526 (+ 1748us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:00.631530 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:00.631533 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:00.631751 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:00.631961 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:00.631962 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:00.631963 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:00.632127 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:00.632401 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:19:00.632405 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:19:00.632518 (+ 113us) client_negotiation.cc:769] Sending connection context
0812 01:19:00.632610 (+ 92us) client_negotiation.cc:240] Negotiation successful
0812 01:19:00.632735 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":85,"threads_started":1}
I20260812 01:19:00.633424 26905 heartbeater.cc:347] Connected to a master server at 127.24.0.190:42265
I20260812 01:19:00.633522 26905 heartbeater.cc:464] Registering TS with master...
I20260812 01:19:00.633706 26905 heartbeater.cc:511] Master 127.24.0.190:42265 requested a full tablet report, sending...
I20260812 01:19:00.634094 26333 ts_manager.cc:194] Registered new tserver with Master: 4ecd5f366111458aa0761a3ed177ae2e (127.24.0.130:38587)
I20260812 01:19:00.634601 26333 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.0.130@KRBTEST.COM'} at 127.24.0.130:42251
I20260812 01:19:00.638283 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 26672
I20260812 01:19:00.638383 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/minicluster-data/ts-1/wal/instance
I20260812 01:19:00.638587 24578 external_mini_cluster.cc:1428] Setting key e9159428c1a9bd2d5a9fb1f77da505d8
I20260812 01:19:00.639631 26912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:00.634397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:42265 (local address 127.24.0.130:48069)
0812 01:19:00.634522 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:00.634532 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:00.634591 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:00.634703 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:00.634703 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:00.634897 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:00.634981 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:00.634983 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:00.635569 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:00.635570 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:00.636041 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:00.636043 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:00.636402 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:00.636403 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:00.636459 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:00.636792 (+ 333us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:00.636796 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:00.637057 (+ 261us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:19:00.638533 (+ 1476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:00.638534 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:00.638534 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:00.638776 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:00.638994 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:00.638995 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:00.638995 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:00.639064 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:00.639393 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:19:00.639394 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:19:00.639431 (+ 37us) client_negotiation.cc:769] Sending connection context
0812 01:19:00.639482 (+ 51us) client_negotiation.cc:240] Negotiation successful
0812 01:19:00.639560 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":75,"threads_started":1}
I20260812 01:19:00.639714 26415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:00.634458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.130:48069 (local address 127.24.0.190:42265)
0812 01:19:00.634495 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:00.634497 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:00.634594 (+ 97us) server_negotiation.cc:407] Connection header received
0812 01:19:00.634625 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:00.634626 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:00.634636 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:00.634700 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:00.635046 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:00.635481 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:00.636154 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:00.636342 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:00.636393 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:00.637128 (+ 735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:00.637129 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:00.637129 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:00.637134 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:00.638454 (+ 1320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:00.638856 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:00.638858 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:00.638858 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:00.638921 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:00.639138 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:00.639138 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:00.639139 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:00.639325 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:00.639407 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:00.639515 (+ 108us) server_negotiation.cc:299] Negotiation successful
0812 01:19:00.639592 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
WARNING: no policy specified for kudu/127.24.0.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.0.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.0.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.131.keytab.
Entry for principal kudu/127.24.0.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.131.keytab.
I20260812 01:19:00.664997 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.0.131:0
--local_ip_for_outbound_sockets=127.24.0.131
--webserver_interface=127.24.0.131
--webserver_port=0
--tserver_master_addrs=127.24.0.190:42265
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.131.keytab
--principal=kudu/127.24.0.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:39799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:19:00.753144 26918 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:19:00.753374 26918 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:19:00.753417 26918 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:19:00.754563 26918 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:19:00.754626 26918 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:19:00.754663 26918 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 01:19:00.754717 26918 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.0.131
I20260812 01:19:00.756223 26918 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:39799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.131.keytab
--principal=kudu/127.24.0.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.0.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.0.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.0.190:42265
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26918
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.0.131
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:19:00.756476 26918 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:19:00.756713 26918 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:19:00.761806 26926 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:00.761828 26925 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:19:00.761859 26918 server_base.cc:1034] running on GCE node
W20260812 01:19:00.761806 26928 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:19:00.762327 26918 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:19:00.762542 26918 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:19:00.763681 26918 hybrid_clock.cc:648] HybridClock initialized: now 1786497540763668 us; error 28 us; skew 500 ppm
Aug 12 01:19:00 dist-test-slave-t3q3 krb5kdc[26269](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497540, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:19:00.766278 26918 init.cc:377] Logged in from keytab as kudu/127.24.0.131@KRBTEST.COM (short username kudu)
I20260812 01:19:00.766723 26918 webserver.cc:460] Webserver started at http://127.24.0.131:32885/ using document root <none> and password file <none>
I20260812 01:19:00.766906 26918 fs_manager.cc:359] Metadata directory not provided
I20260812 01:19:00.766947 26918 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:19:00.767032 26918 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:19:00.768008 26918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/minicluster-data/ts-2/data/instance:
uuid: "2fc33b8fef874582803e886295a87edb"
format_stamp: "Formatted at 2026-08-12 01:19:00 on dist-test-slave-t3q3"
server_key: "aac94d29a533fe73000e011064417a7d"
server_key_iv: "11a8a9de66f548ea4408e38a21a1b5b3"
server_key_version: "encryptionkey@0"
I20260812 01:19:00.768379 26918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/minicluster-data/ts-2/wal/instance:
uuid: "2fc33b8fef874582803e886295a87edb"
format_stamp: "Formatted at 2026-08-12 01:19:00 on dist-test-slave-t3q3"
server_key: "aac94d29a533fe73000e011064417a7d"
server_key_iv: "11a8a9de66f548ea4408e38a21a1b5b3"
server_key_version: "encryptionkey@0"
I20260812 01:19:00.769650 26918 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:19:00.770394 26939 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:00.770716 26918 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:19:00.770821 26918 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/minicluster-data/ts-2/wal
uuid: "2fc33b8fef874582803e886295a87edb"
format_stamp: "Formatted at 2026-08-12 01:19:00 on dist-test-slave-t3q3"
server_key: "aac94d29a533fe73000e011064417a7d"
server_key_iv: "11a8a9de66f548ea4408e38a21a1b5b3"
server_key_version: "encryptionkey@0"
I20260812 01:19:00.770877 26918 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:19:00.792660 26918 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:19:00.795024 26918 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:19:00.795140 26918 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:19:00.795624 26918 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:19:00.795665 26918 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:00.795689 26918 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:19:00.795747 26918 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 01:19:00 dist-test-slave-t3q3 krb5kdc[26269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497540, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.131@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
I20260812 01:19:00.803336 26918 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.131:34543
I20260812 01:19:00.803386 27142 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.131:34543 every 8 connection(s)
I20260812 01:19:00.803710 26918 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/minicluster-data/ts-2/data/info.pb
I20260812 01:19:00.806466 26415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:00.796163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.131:49723 (local address 127.24.0.190:42265)
0812 01:19:00.796237 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:00.796239 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:00.796503 (+ 264us) server_negotiation.cc:407] Connection header received
0812 01:19:00.797259 (+ 756us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:00.797260 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:00.797271 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:00.797317 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:00.798243 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:00.799049 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:00.800762 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:00.801084 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:00.801162 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:00.803851 (+ 2689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:00.803852 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:00.803853 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:00.803861 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:00.805365 (+ 1504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:00.805727 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:00.805728 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:00.805728 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:00.805780 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:00.805952 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:00.805952 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:00.805953 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:00.806055 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:00.806105 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:00.806336 (+ 231us) server_negotiation.cc:299] Negotiation successful
0812 01:19:00.806401 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 01:19:00.806618 26955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:00.796139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:42265 (local address 127.24.0.131:49723)
0812 01:19:00.796353 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:00.796407 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:00.797134 (+ 727us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:00.797404 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:00.797409 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:00.797825 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:00.798137 (+ 312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:00.798145 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:00.799160 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:00.799161 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:00.800670 (+ 1509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:00.800674 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:00.801170 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:00.801171 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:00.801335 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:00.801913 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:00.801927 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:00.803724 (+ 1797us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:19:00.805441 (+ 1717us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:00.805445 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:00.805447 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:00.805659 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:00.805822 (+ 163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:00.805822 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:00.805823 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:00.805906 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:00.806111 (+ 205us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:19:00.806116 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:19:00.806230 (+ 114us) client_negotiation.cc:769] Sending connection context
0812 01:19:00.806313 (+ 83us) client_negotiation.cc:240] Negotiation successful
0812 01:19:00.806431 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":112,"threads_started":1}
I20260812 01:19:00.807085 27146 heartbeater.cc:347] Connected to a master server at 127.24.0.190:42265
I20260812 01:19:00.807194 27146 heartbeater.cc:464] Registering TS with master...
I20260812 01:19:00.807364 27146 heartbeater.cc:511] Master 127.24.0.190:42265 requested a full tablet report, sending...
I20260812 01:19:00.807785 26334 ts_manager.cc:194] Registered new tserver with Master: 2fc33b8fef874582803e886295a87edb (127.24.0.131:34543)
I20260812 01:19:00.808349 26334 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.0.131@KRBTEST.COM'} at 127.24.0.131:49723
I20260812 01:19:00.810576 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 26918
I20260812 01:19:00.810689 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/minicluster-data/ts-2/wal/instance
I20260812 01:19:00.810926 24578 external_mini_cluster.cc:1428] Setting key 80e367038f19d4592a242b3a4e6b5057
I20260812 01:19:00.811944 24578 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 01:19:00.814092 26415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:00.808296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.131:58865 (local address 127.24.0.190:42265)
0812 01:19:00.808336 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:00.808338 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:00.808599 (+ 261us) server_negotiation.cc:407] Connection header received
0812 01:19:00.808685 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:00.808685 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:00.808697 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:00.808751 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:00.809074 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:00.809581 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:00.810251 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:00.810423 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:00.810479 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:00.811527 (+ 1048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:00.811528 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:00.811528 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:00.811536 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:00.812969 (+ 1433us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:00.813331 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:00.813332 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:00.813333 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:00.813393 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:00.813585 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:00.813586 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:00.813586 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:00.813696 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:00.813770 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:00.813923 (+ 153us) server_negotiation.cc:299] Negotiation successful
0812 01:19:00.813998 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 01:19:00.814097 27159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:00.808211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:42265 (local address 127.24.0.131:58865)
0812 01:19:00.808534 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:00.808546 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:00.808614 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:00.808768 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:00.808769 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:00.808928 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:00.808982 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:00.808983 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:00.809671 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:00.809672 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:00.810173 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:00.810175 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:00.810499 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:00.810500 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:00.810585 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:00.811048 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:00.811053 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:00.811438 (+ 385us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:19:00.813064 (+ 1626us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:00.813065 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:00.813065 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:00.813246 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:00.813457 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:00.813458 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:00.813459 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:00.813519 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:00.813763 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:19:00.813764 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:19:00.813810 (+ 46us) client_negotiation.cc:769] Sending connection context
0812 01:19:00.813894 (+ 84us) client_negotiation.cc:240] Negotiation successful
0812 01:19:00.814000 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":81,"threads_started":1}
Aug 12 01:19:00 dist-test-slave-t3q3 krb5kdc[26269](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497540, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 01:19:00 dist-test-slave-t3q3 krb5kdc[26269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497540, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
I20260812 01:19:00.837462 26415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:00.830797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59884 (local address 127.24.0.190:42265)
0812 01:19:00.830862 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:00.830864 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:00.830873 (+ 9us) server_negotiation.cc:407] Connection header received
0812 01:19:00.830899 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:00.830900 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:00.830910 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:00.830970 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:00.831529 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:00.832012 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:00.832751 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:00.832930 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:00.832979 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:00.835297 (+ 2318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:00.835298 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:00.835298 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:00.835304 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:00.836552 (+ 1248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:00.836865 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:00.836866 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:00.836866 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:00.836915 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:00.837073 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:00.837074 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:00.837074 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:00.837170 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:00.837249 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:00.837323 (+ 74us) server_negotiation.cc:299] Negotiation successful
0812 01:19:00.837378 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 01:19:00.839140 26334 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59884:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 01:19:00.840808 26334 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 01:19:00.847318 27180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:00.843346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50958 (local address 127.24.0.129:38473)
0812 01:19:00.843664 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:00.843668 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:00.843676 (+ 8us) server_negotiation.cc:407] Connection header received
0812 01:19:00.843765 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:00.843768 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:00.843785 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:00.843873 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:19:00.844037 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:00.844948 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:00.846388 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:00.846897 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:00.846985 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:00.847048 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:00.847190 (+ 142us) server_negotiation.cc:299] Negotiation successful
0812 01:19:00.847228 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":280,"thread_start_us":70,"threads_started":1}
I20260812 01:19:00.847313 27179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:00.843128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.129:38473 (local address 127.0.0.1:50958)
0812 01:19:00.843465 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:00.843477 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:00.843537 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:00.843872 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:00.843873 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:00.843875 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:19:00.843933 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:00.843934 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:00.845033 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:00.845034 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:00.846299 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:00.846302 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:00.846985 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:00.846986 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:00.847067 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:00.847081 (+ 14us) client_negotiation.cc:769] Sending connection context
0812 01:19:00.847170 (+ 89us) client_negotiation.cc:240] Negotiation successful
0812 01:19:00.847225 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"mutex_wait_us":4,"thread_start_us":62,"threads_started":1}
I20260812 01:19:00.847530 27176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:00.843127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.130:38587 (local address 127.0.0.1:48922)
0812 01:19:00.843299 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:00.843340 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:00.843407 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:00.843776 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:00.843779 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:00.843782 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:19:00.843854 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:00.843856 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:00.845322 (+ 1466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:00.845323 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:00.846380 (+ 1057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:00.846382 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:00.847288 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:00.847289 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:00.847368 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:00.847371 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 01:19:00.847403 (+ 32us) client_negotiation.cc:240] Negotiation successful
0812 01:19:00.847450 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":81,"threads_started":1}
I20260812 01:19:00.847550 27175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:00.843329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.131:34543 (local address 127.0.0.1:38244)
0812 01:19:00.843467 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:00.843476 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:00.843520 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:00.844195 (+ 675us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:00.844196 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:00.844204 (+ 8us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:19:00.844238 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:00.844239 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:00.845441 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:00.845442 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:00.846334 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:00.846337 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:00.847345 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:00.847346 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:00.847402 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:00.847406 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 01:19:00.847451 (+ 45us) client_negotiation.cc:240] Negotiation successful
0812 01:19:00.847480 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"mutex_wait_us":1,"thread_start_us":89,"threads_started":1}
I20260812 01:19:00.847813 27184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:00.843599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38244 (local address 127.24.0.131:34543)
0812 01:19:00.843792 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:00.843796 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:00.843806 (+ 10us) server_negotiation.cc:407] Connection header received
0812 01:19:00.844044 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:00.844047 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:00.844095 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:00.844186 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:19:00.844305 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:00.845358 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:00.846436 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:00.847180 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:00.847260 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:00.847317 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:00.847589 (+ 272us) server_negotiation.cc:299] Negotiation successful
0812 01:19:00.847717 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":112,"threads_started":1}
I20260812 01:19:00.847920 27177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:00.843222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48922 (local address 127.24.0.130:38587)
0812 01:19:00.843570 (+ 348us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:00.843575 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:00.843586 (+ 11us) server_negotiation.cc:407] Connection header received
0812 01:19:00.843628 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:00.843630 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:00.843648 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:00.843943 (+ 295us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:19:00.844015 (+ 72us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:00.845232 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:00.846471 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:00.847180 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:00.847672 (+ 492us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:00.847747 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:00.847774 (+ 27us) server_negotiation.cc:299] Negotiation successful
0812 01:19:00.847813 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":119,"threads_started":1}
I20260812 01:19:00.848510 27004 tablet_service.cc:1467] Processing CreateTablet for tablet 386a067d9c73409ba12f9f1a5d2d84de (DEFAULT_TABLE table=test-table [id=858080bbcebd43759db8edaf5c7a1e83]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:19:00.848567 26552 tablet_service.cc:1467] Processing CreateTablet for tablet 386a067d9c73409ba12f9f1a5d2d84de (DEFAULT_TABLE table=test-table [id=858080bbcebd43759db8edaf5c7a1e83]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:19:00.848567 26795 tablet_service.cc:1467] Processing CreateTablet for tablet 386a067d9c73409ba12f9f1a5d2d84de (DEFAULT_TABLE table=test-table [id=858080bbcebd43759db8edaf5c7a1e83]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:19:00.848876 27004 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 386a067d9c73409ba12f9f1a5d2d84de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:19:00.848923 26795 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 386a067d9c73409ba12f9f1a5d2d84de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:19:00.848923 26552 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 386a067d9c73409ba12f9f1a5d2d84de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:19:00.851307 27185 tablet_bootstrap.cc:492] T 386a067d9c73409ba12f9f1a5d2d84de P 2fc33b8fef874582803e886295a87edb: Bootstrap starting.
I20260812 01:19:00.852301 27185 tablet_bootstrap.cc:654] T 386a067d9c73409ba12f9f1a5d2d84de P 2fc33b8fef874582803e886295a87edb: Neither blocks nor log segments found. Creating new log.
I20260812 01:19:00.852628 27186 tablet_bootstrap.cc:492] T 386a067d9c73409ba12f9f1a5d2d84de P 6944457d5a3046a2b3623fc868ee3aa1: Bootstrap starting.
I20260812 01:19:00.852694 27185 log.cc:824] T 386a067d9c73409ba12f9f1a5d2d84de P 2fc33b8fef874582803e886295a87edb: Log is configured to *not* fsync() on all Append() calls
I20260812 01:19:00.853104 27189 tablet_bootstrap.cc:492] T 386a067d9c73409ba12f9f1a5d2d84de P 4ecd5f366111458aa0761a3ed177ae2e: Bootstrap starting.
I20260812 01:19:00.853384 27186 tablet_bootstrap.cc:654] T 386a067d9c73409ba12f9f1a5d2d84de P 6944457d5a3046a2b3623fc868ee3aa1: Neither blocks nor log segments found. Creating new log.
I20260812 01:19:00.853488 27185 tablet_bootstrap.cc:492] T 386a067d9c73409ba12f9f1a5d2d84de P 2fc33b8fef874582803e886295a87edb: No bootstrap required, opened a new log
I20260812 01:19:00.853564 27185 ts_tablet_manager.cc:1397] T 386a067d9c73409ba12f9f1a5d2d84de P 2fc33b8fef874582803e886295a87edb: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 01:19:00.853734 27186 log.cc:824] T 386a067d9c73409ba12f9f1a5d2d84de P 6944457d5a3046a2b3623fc868ee3aa1: Log is configured to *not* fsync() on all Append() calls
I20260812 01:19:00.853765 27189 tablet_bootstrap.cc:654] T 386a067d9c73409ba12f9f1a5d2d84de P 4ecd5f366111458aa0761a3ed177ae2e: Neither blocks nor log segments found. Creating new log.
I20260812 01:19:00.854080 27189 log.cc:824] T 386a067d9c73409ba12f9f1a5d2d84de P 4ecd5f366111458aa0761a3ed177ae2e: Log is configured to *not* fsync() on all Append() calls
I20260812 01:19:00.854518 27186 tablet_bootstrap.cc:492] T 386a067d9c73409ba12f9f1a5d2d84de P 6944457d5a3046a2b3623fc868ee3aa1: No bootstrap required, opened a new log
I20260812 01:19:00.854594 27186 ts_tablet_manager.cc:1397] T 386a067d9c73409ba12f9f1a5d2d84de P 6944457d5a3046a2b3623fc868ee3aa1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 01:19:00.854825 27189 tablet_bootstrap.cc:492] T 386a067d9c73409ba12f9f1a5d2d84de P 4ecd5f366111458aa0761a3ed177ae2e: No bootstrap required, opened a new log
I20260812 01:19:00.854892 27189 ts_tablet_manager.cc:1397] T 386a067d9c73409ba12f9f1a5d2d84de P 4ecd5f366111458aa0761a3ed177ae2e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 01:19:00.855535 27185 raft_consensus.cc:348] T 386a067d9c73409ba12f9f1a5d2d84de P 2fc33b8fef874582803e886295a87edb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6944457d5a3046a2b3623fc868ee3aa1" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 38473 } } peers { permanent_uuid: "2fc33b8fef874582803e886295a87edb" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34543 } } peers { permanent_uuid: "4ecd5f366111458aa0761a3ed177ae2e" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 38587 } }
I20260812 01:19:00.855716 27185 raft_consensus.cc:374] T 386a067d9c73409ba12f9f1a5d2d84de P 2fc33b8fef874582803e886295a87edb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:19:00.855764 27185 raft_consensus.cc:729] T 386a067d9c73409ba12f9f1a5d2d84de P 2fc33b8fef874582803e886295a87edb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fc33b8fef874582803e886295a87edb, State: Initialized, Role: FOLLOWER
I20260812 01:19:00.855899 27185 consensus_queue.cc:260] T 386a067d9c73409ba12f9f1a5d2d84de P 2fc33b8fef874582803e886295a87edb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6944457d5a3046a2b3623fc868ee3aa1" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 38473 } } peers { permanent_uuid: "2fc33b8fef874582803e886295a87edb" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34543 } } peers { permanent_uuid: "4ecd5f366111458aa0761a3ed177ae2e" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 38587 } }
I20260812 01:19:00.856206 27185 ts_tablet_manager.cc:1428] T 386a067d9c73409ba12f9f1a5d2d84de P 2fc33b8fef874582803e886295a87edb: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 01:19:00.856228 27146 heartbeater.cc:503] Master 127.24.0.190:42265 was elected leader, sending a full tablet report...
I20260812 01:19:00.856616 27186 raft_consensus.cc:348] T 386a067d9c73409ba12f9f1a5d2d84de P 6944457d5a3046a2b3623fc868ee3aa1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6944457d5a3046a2b3623fc868ee3aa1" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 38473 } } peers { permanent_uuid: "2fc33b8fef874582803e886295a87edb" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34543 } } peers { permanent_uuid: "4ecd5f366111458aa0761a3ed177ae2e" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 38587 } }
I20260812 01:19:00.856779 27186 raft_consensus.cc:374] T 386a067d9c73409ba12f9f1a5d2d84de P 6944457d5a3046a2b3623fc868ee3aa1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:19:00.856824 27186 raft_consensus.cc:729] T 386a067d9c73409ba12f9f1a5d2d84de P 6944457d5a3046a2b3623fc868ee3aa1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6944457d5a3046a2b3623fc868ee3aa1, State: Initialized, Role: FOLLOWER
I20260812 01:19:00.856950 27186 consensus_queue.cc:260] T 386a067d9c73409ba12f9f1a5d2d84de P 6944457d5a3046a2b3623fc868ee3aa1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6944457d5a3046a2b3623fc868ee3aa1" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 38473 } } peers { permanent_uuid: "2fc33b8fef874582803e886295a87edb" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34543 } } peers { permanent_uuid: "4ecd5f366111458aa0761a3ed177ae2e" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 38587 } }
I20260812 01:19:00.857223 27186 ts_tablet_manager.cc:1428] T 386a067d9c73409ba12f9f1a5d2d84de P 6944457d5a3046a2b3623fc868ee3aa1: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 01:19:00.857273 26655 heartbeater.cc:503] Master 127.24.0.190:42265 was elected leader, sending a full tablet report...
I20260812 01:19:00.857223 27189 raft_consensus.cc:348] T 386a067d9c73409ba12f9f1a5d2d84de P 4ecd5f366111458aa0761a3ed177ae2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6944457d5a3046a2b3623fc868ee3aa1" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 38473 } } peers { permanent_uuid: "2fc33b8fef874582803e886295a87edb" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34543 } } peers { permanent_uuid: "4ecd5f366111458aa0761a3ed177ae2e" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 38587 } }
I20260812 01:19:00.857367 27189 raft_consensus.cc:374] T 386a067d9c73409ba12f9f1a5d2d84de P 4ecd5f366111458aa0761a3ed177ae2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:19:00.857414 27189 raft_consensus.cc:729] T 386a067d9c73409ba12f9f1a5d2d84de P 4ecd5f366111458aa0761a3ed177ae2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ecd5f366111458aa0761a3ed177ae2e, State: Initialized, Role: FOLLOWER
I20260812 01:19:00.857522 27189 consensus_queue.cc:260] T 386a067d9c73409ba12f9f1a5d2d84de P 4ecd5f366111458aa0761a3ed177ae2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6944457d5a3046a2b3623fc868ee3aa1" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 38473 } } peers { permanent_uuid: "2fc33b8fef874582803e886295a87edb" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34543 } } peers { permanent_uuid: "4ecd5f366111458aa0761a3ed177ae2e" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 38587 } }
I20260812 01:19:00.857797 27189 ts_tablet_manager.cc:1428] T 386a067d9c73409ba12f9f1a5d2d84de P 4ecd5f366111458aa0761a3ed177ae2e: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260812 01:19:00.857826 26905 heartbeater.cc:503] Master 127.24.0.190:42265 was elected leader, sending a full tablet report...
I20260812 01:19:00.870151 27199 raft_consensus.cc:482] T 386a067d9c73409ba12f9f1a5d2d84de P 2fc33b8fef874582803e886295a87edb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 01:19:00.870268 27199 raft_consensus.cc:504] T 386a067d9c73409ba12f9f1a5d2d84de P 2fc33b8fef874582803e886295a87edb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6944457d5a3046a2b3623fc868ee3aa1" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 38473 } } peers { permanent_uuid: "2fc33b8fef874582803e886295a87edb" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34543 } } peers { permanent_uuid: "4ecd5f366111458aa0761a3ed177ae2e" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 38587 } }
I20260812 01:19:00.870579 27199 leader_election.cc:288] T 386a067d9c73409ba12f9f1a5d2d84de P 2fc33b8fef874582803e886295a87edb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6944457d5a3046a2b3623fc868ee3aa1 (127.24.0.129:38473), 4ecd5f366111458aa0761a3ed177ae2e (127.24.0.130:38587)
I20260812 01:19:00.873775 27204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:00.870791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.129:38473 (local address 127.24.0.131:46519)
0812 01:19:00.870989 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:00.870999 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:00.871101 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:00.871263 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:00.871264 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:00.871266 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:19:00.871330 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:00.871332 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:00.871929 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:00.871931 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:00.872896 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:00.872900 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:00.873466 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:00.873467 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:00.873568 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:00.873573 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 01:19:00.873643 (+ 70us) client_negotiation.cc:240] Negotiation successful
0812 01:19:00.873690 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"mutex_wait_us":3,"thread_start_us":119,"threads_started":1}
I20260812 01:19:00.873771 27180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:00.870878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.131:46519 (local address 127.24.0.129:38473)
0812 01:19:00.870921 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:00.870923 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:00.871085 (+ 162us) server_negotiation.cc:407] Connection header received
0812 01:19:00.871157 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:00.871159 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:00.871171 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:00.871240 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:19:00.871396 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:00.871830 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:00.872999 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:00.873398 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:00.873451 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:00.873457 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:00.873660 (+ 203us) server_negotiation.cc:299] Negotiation successful
0812 01:19:00.873696 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 01:19:00.873965 26955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:00.870781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.130:38587 (local address 127.24.0.131:60587)
0812 01:19:00.870837 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:00.870850 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:00.870892 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:00.871091 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:00.871092 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:00.871094 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:19:00.871164 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:00.871167 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:00.872158 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:00.872159 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:00.873054 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:00.873057 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:00.873680 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:00.873681 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:00.873772 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:00.873776 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 01:19:00.873838 (+ 62us) client_negotiation.cc:240] Negotiation successful
0812 01:19:00.873894 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260812 01:19:00.874166 26584 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "386a067d9c73409ba12f9f1a5d2d84de" candidate_uuid: "2fc33b8fef874582803e886295a87edb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6944457d5a3046a2b3623fc868ee3aa1" is_pre_election: true
I20260812 01:19:00.874279 27177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:00.870880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.131:60587 (local address 127.24.0.130:38587)
0812 01:19:00.870963 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:00.870965 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:00.870974 (+ 9us) server_negotiation.cc:407] Connection header received
0812 01:19:00.871001 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:00.871002 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:00.871014 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:00.871075 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:19:00.871244 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:00.871959 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:00.873134 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:00.873613 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:00.873677 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:00.873684 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:00.874176 (+ 492us) server_negotiation.cc:299] Negotiation successful
0812 01:19:00.874217 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 01:19:00.874336 26584 raft_consensus.cc:2454] T 386a067d9c73409ba12f9f1a5d2d84de P 6944457d5a3046a2b3623fc868ee3aa1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2fc33b8fef874582803e886295a87edb in term 0.
I20260812 01:19:00.874529 26832 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "386a067d9c73409ba12f9f1a5d2d84de" candidate_uuid: "2fc33b8fef874582803e886295a87edb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ecd5f366111458aa0761a3ed177ae2e" is_pre_election: true
I20260812 01:19:00.874645 26940 leader_election.cc:302] T 386a067d9c73409ba12f9f1a5d2d84de P 2fc33b8fef874582803e886295a87edb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2fc33b8fef874582803e886295a87edb, 6944457d5a3046a2b3623fc868ee3aa1; no voters:
I20260812 01:19:00.874670 26832 raft_consensus.cc:2454] T 386a067d9c73409ba12f9f1a5d2d84de P 4ecd5f366111458aa0761a3ed177ae2e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2fc33b8fef874582803e886295a87edb in term 0.
I20260812 01:19:00.874766 27199 raft_consensus.cc:2781] T 386a067d9c73409ba12f9f1a5d2d84de P 2fc33b8fef874582803e886295a87edb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 01:19:00.874830 27199 raft_consensus.cc:482] T 386a067d9c73409ba12f9f1a5d2d84de P 2fc33b8fef874582803e886295a87edb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 01:19:00.874866 27199 raft_consensus.cc:3037] T 386a067d9c73409ba12f9f1a5d2d84de P 2fc33b8fef874582803e886295a87edb [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:19:00.875550 27199 raft_consensus.cc:504] T 386a067d9c73409ba12f9f1a5d2d84de P 2fc33b8fef874582803e886295a87edb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6944457d5a3046a2b3623fc868ee3aa1" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 38473 } } peers { permanent_uuid: "2fc33b8fef874582803e886295a87edb" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34543 } } peers { permanent_uuid: "4ecd5f366111458aa0761a3ed177ae2e" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 38587 } }
I20260812 01:19:00.875677 27199 leader_election.cc:288] T 386a067d9c73409ba12f9f1a5d2d84de P 2fc33b8fef874582803e886295a87edb [CANDIDATE]: Term 1 election: Requested vote from peers 6944457d5a3046a2b3623fc868ee3aa1 (127.24.0.129:38473), 4ecd5f366111458aa0761a3ed177ae2e (127.24.0.130:38587)
I20260812 01:19:00.875833 26584 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "386a067d9c73409ba12f9f1a5d2d84de" candidate_uuid: "2fc33b8fef874582803e886295a87edb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6944457d5a3046a2b3623fc868ee3aa1"
I20260812 01:19:00.875880 26832 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "386a067d9c73409ba12f9f1a5d2d84de" candidate_uuid: "2fc33b8fef874582803e886295a87edb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ecd5f366111458aa0761a3ed177ae2e"
I20260812 01:19:00.875901 26584 raft_consensus.cc:3037] T 386a067d9c73409ba12f9f1a5d2d84de P 6944457d5a3046a2b3623fc868ee3aa1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:19:00.875955 26832 raft_consensus.cc:3037] T 386a067d9c73409ba12f9f1a5d2d84de P 4ecd5f366111458aa0761a3ed177ae2e [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:19:00.876632 26584 raft_consensus.cc:2454] T 386a067d9c73409ba12f9f1a5d2d84de P 6944457d5a3046a2b3623fc868ee3aa1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2fc33b8fef874582803e886295a87edb in term 1.
I20260812 01:19:00.876678 26832 raft_consensus.cc:2454] T 386a067d9c73409ba12f9f1a5d2d84de P 4ecd5f366111458aa0761a3ed177ae2e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2fc33b8fef874582803e886295a87edb in term 1.
I20260812 01:19:00.877432 26942 leader_election.cc:302] T 386a067d9c73409ba12f9f1a5d2d84de P 2fc33b8fef874582803e886295a87edb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2fc33b8fef874582803e886295a87edb, 4ecd5f366111458aa0761a3ed177ae2e; no voters:
I20260812 01:19:00.877586 27199 raft_consensus.cc:2781] T 386a067d9c73409ba12f9f1a5d2d84de P 2fc33b8fef874582803e886295a87edb [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:19:00.877731 27199 raft_consensus.cc:686] T 386a067d9c73409ba12f9f1a5d2d84de P 2fc33b8fef874582803e886295a87edb [term 1 LEADER]: Becoming Leader. State: Replica: 2fc33b8fef874582803e886295a87edb, State: Running, Role: LEADER
I20260812 01:19:00.877825 27199 consensus_queue.cc:237] T 386a067d9c73409ba12f9f1a5d2d84de P 2fc33b8fef874582803e886295a87edb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6944457d5a3046a2b3623fc868ee3aa1" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 38473 } } peers { permanent_uuid: "2fc33b8fef874582803e886295a87edb" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34543 } } peers { permanent_uuid: "4ecd5f366111458aa0761a3ed177ae2e" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 38587 } }
I20260812 01:19:00.878639 26334 catalog_manager.cc:5495] T 386a067d9c73409ba12f9f1a5d2d84de P 2fc33b8fef874582803e886295a87edb reported cstate change: term changed from 0 to 1, leader changed from <none> to 2fc33b8fef874582803e886295a87edb (127.24.0.131). New cstate: current_term: 1 leader_uuid: "2fc33b8fef874582803e886295a87edb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6944457d5a3046a2b3623fc868ee3aa1" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 38473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2fc33b8fef874582803e886295a87edb" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34543 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4ecd5f366111458aa0761a3ed177ae2e" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 38587 } health_report { overall_health: UNKNOWN } } }
W20260812 01:19:00.882318 26907 tablet.cc:2369] T 386a067d9c73409ba12f9f1a5d2d84de P 4ecd5f366111458aa0761a3ed177ae2e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 01:19:00.887185 27184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:00.884963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38256 (local address 127.24.0.131:34543)
0812 01:19:00.885001 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:00.885003 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:00.885025 (+ 22us) server_negotiation.cc:407] Connection header received
0812 01:19:00.885098 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:00.885099 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:00.885114 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:00.885179 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:19:00.885316 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:00.885749 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:00.886477 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:00.886658 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:00.886712 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:00.886886 (+ 174us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:19:00.886971 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:19:00.887070 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:00.887103 (+ 33us) server_negotiation.cc:299] Negotiation successful
0812 01:19:00.887120 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 01:19:00.888926 26832 raft_consensus.cc:1261] T 386a067d9c73409ba12f9f1a5d2d84de P 4ecd5f366111458aa0761a3ed177ae2e [term 1 FOLLOWER]: Refusing update from remote peer 2fc33b8fef874582803e886295a87edb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:19:00.888926 26584 raft_consensus.cc:1261] T 386a067d9c73409ba12f9f1a5d2d84de P 6944457d5a3046a2b3623fc868ee3aa1 [term 1 FOLLOWER]: Refusing update from remote peer 2fc33b8fef874582803e886295a87edb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:19:00.889389 27207 consensus_queue.cc:1035] T 386a067d9c73409ba12f9f1a5d2d84de P 2fc33b8fef874582803e886295a87edb [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ecd5f366111458aa0761a3ed177ae2e" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 38587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:19:00.889562 27199 consensus_queue.cc:1035] T 386a067d9c73409ba12f9f1a5d2d84de P 2fc33b8fef874582803e886295a87edb [LEADER]: Connected to new peer: Peer: permanent_uuid: "6944457d5a3046a2b3623fc868ee3aa1" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 38473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:19:00.892652 27214 mvcc.cc:205] Tried to move back new op lower bound from 7317493927478329344 to 7317493927436263424. Current Snapshot: MvccSnapshot[applied={T|T < 7317493927478329344}]
I20260812 01:19:00.892953 27213 mvcc.cc:205] Tried to move back new op lower bound from 7317493927478329344 to 7317493927436263424. Current Snapshot: MvccSnapshot[applied={T|T < 7317493927478329344}]
I20260812 01:19:00.895710 26333 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59884:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:19:00.895850 26333 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59884:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:19:00.896868 26333 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dec86a9c3f9845a383f2cc0f2a31b340: Sending DeleteTablet for 3 replicas of tablet 386a067d9c73409ba12f9f1a5d2d84de
I20260812 01:19:00.897353 26552 tablet_service.cc:1514] Processing DeleteTablet for tablet 386a067d9c73409ba12f9f1a5d2d84de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:19:00 UTC) from {username='kudu', principal='kudu/127.24.0.190@KRBTEST.COM'} at 127.0.0.1:50958
I20260812 01:19:00.897342 26795 tablet_service.cc:1514] Processing DeleteTablet for tablet 386a067d9c73409ba12f9f1a5d2d84de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:19:00 UTC) from {username='kudu', principal='kudu/127.24.0.190@KRBTEST.COM'} at 127.0.0.1:48922
I20260812 01:19:00.897581 27004 tablet_service.cc:1514] Processing DeleteTablet for tablet 386a067d9c73409ba12f9f1a5d2d84de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:19:00 UTC) from {username='kudu', principal='kudu/127.24.0.190@KRBTEST.COM'} at 127.0.0.1:38244
I20260812 01:19:00.897543 26333 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59884:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 01:19:00.897706 26333 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 01:19:00.897799 27229 tablet_replica.cc:330] T 386a067d9c73409ba12f9f1a5d2d84de P 2fc33b8fef874582803e886295a87edb: stopping tablet replica
I20260812 01:19:00.897890 27229 raft_consensus.cc:2229] T 386a067d9c73409ba12f9f1a5d2d84de P 2fc33b8fef874582803e886295a87edb [term 1 LEADER]: Raft consensus shutting down.
I20260812 01:19:00.897974 27229 raft_consensus.cc:2258] T 386a067d9c73409ba12f9f1a5d2d84de P 2fc33b8fef874582803e886295a87edb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:19:00.898422 27229 ts_tablet_manager.cc:1905] T 386a067d9c73409ba12f9f1a5d2d84de P 2fc33b8fef874582803e886295a87edb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:19:00.899060 27004 tablet_service.cc:1467] Processing CreateTablet for tablet 6297467749854de8b8f2350156052bb2 (DEFAULT_TABLE table=test-table [id=8a26d5ec9ce64104bc0deb7be207aad8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:19:00.899114 26552 tablet_service.cc:1467] Processing CreateTablet for tablet 6297467749854de8b8f2350156052bb2 (DEFAULT_TABLE table=test-table [id=8a26d5ec9ce64104bc0deb7be207aad8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:19:00.899194 27004 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6297467749854de8b8f2350156052bb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:19:00.899215 26552 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6297467749854de8b8f2350156052bb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:19:00.899410 26795 tablet_service.cc:1467] Processing CreateTablet for tablet 6297467749854de8b8f2350156052bb2 (DEFAULT_TABLE table=test-table [id=8a26d5ec9ce64104bc0deb7be207aad8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:19:00.899571 26795 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6297467749854de8b8f2350156052bb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:19:00.900642 27185 tablet_bootstrap.cc:492] T 6297467749854de8b8f2350156052bb2 P 2fc33b8fef874582803e886295a87edb: Bootstrap starting.
I20260812 01:19:00.900810 27186 tablet_bootstrap.cc:492] T 6297467749854de8b8f2350156052bb2 P 6944457d5a3046a2b3623fc868ee3aa1: Bootstrap starting.
I20260812 01:19:00.900827 27189 tablet_bootstrap.cc:492] T 6297467749854de8b8f2350156052bb2 P 4ecd5f366111458aa0761a3ed177ae2e: Bootstrap starting.
I20260812 01:19:00.901113 27185 tablet_bootstrap.cc:654] T 6297467749854de8b8f2350156052bb2 P 2fc33b8fef874582803e886295a87edb: Neither blocks nor log segments found. Creating new log.
I20260812 01:19:00.901235 27186 tablet_bootstrap.cc:654] T 6297467749854de8b8f2350156052bb2 P 6944457d5a3046a2b3623fc868ee3aa1: Neither blocks nor log segments found. Creating new log.
I20260812 01:19:00.901271 27189 tablet_bootstrap.cc:654] T 6297467749854de8b8f2350156052bb2 P 4ecd5f366111458aa0761a3ed177ae2e: Neither blocks nor log segments found. Creating new log.
I20260812 01:19:00.901808 27185 tablet_bootstrap.cc:492] T 6297467749854de8b8f2350156052bb2 P 2fc33b8fef874582803e886295a87edb: No bootstrap required, opened a new log
I20260812 01:19:00.901880 27185 ts_tablet_manager.cc:1397] T 6297467749854de8b8f2350156052bb2 P 2fc33b8fef874582803e886295a87edb: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260812 01:19:00.902070 27185 raft_consensus.cc:348] T 6297467749854de8b8f2350156052bb2 P 2fc33b8fef874582803e886295a87edb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6944457d5a3046a2b3623fc868ee3aa1" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 38473 } } peers { permanent_uuid: "4ecd5f366111458aa0761a3ed177ae2e" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 38587 } } peers { permanent_uuid: "2fc33b8fef874582803e886295a87edb" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34543 } }
I20260812 01:19:00.902151 27185 raft_consensus.cc:374] T 6297467749854de8b8f2350156052bb2 P 2fc33b8fef874582803e886295a87edb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:19:00.902181 27185 raft_consensus.cc:729] T 6297467749854de8b8f2350156052bb2 P 2fc33b8fef874582803e886295a87edb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fc33b8fef874582803e886295a87edb, State: Initialized, Role: FOLLOWER
I20260812 01:19:00.902235 27185 consensus_queue.cc:260] T 6297467749854de8b8f2350156052bb2 P 2fc33b8fef874582803e886295a87edb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6944457d5a3046a2b3623fc868ee3aa1" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 38473 } } peers { permanent_uuid: "4ecd5f366111458aa0761a3ed177ae2e" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 38587 } } peers { permanent_uuid: "2fc33b8fef874582803e886295a87edb" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34543 } }
I20260812 01:19:00.902320 27185 ts_tablet_manager.cc:1428] T 6297467749854de8b8f2350156052bb2 P 2fc33b8fef874582803e886295a87edb: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260812 01:19:00.903206 27229 ts_tablet_manager.cc:1918] T 386a067d9c73409ba12f9f1a5d2d84de P 2fc33b8fef874582803e886295a87edb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 01:19:00.903285 27229 log.cc:1196] T 386a067d9c73409ba12f9f1a5d2d84de P 2fc33b8fef874582803e886295a87edb: Deleting WAL directory at /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/minicluster-data/ts-2/wal/wals/386a067d9c73409ba12f9f1a5d2d84de
I20260812 01:19:00.903724 27229 ts_tablet_manager.cc:1939] T 386a067d9c73409ba12f9f1a5d2d84de P 2fc33b8fef874582803e886295a87edb: Deleting consensus metadata
I20260812 01:19:00.904291 26308 catalog_manager.cc:4841] TS 2fc33b8fef874582803e886295a87edb (127.24.0.131:34543): tablet 386a067d9c73409ba12f9f1a5d2d84de (table test-table [id=858080bbcebd43759db8edaf5c7a1e83]) successfully deleted
I20260812 01:19:00.904474 27186 tablet_bootstrap.cc:492] T 6297467749854de8b8f2350156052bb2 P 6944457d5a3046a2b3623fc868ee3aa1: No bootstrap required, opened a new log
I20260812 01:19:00.904531 27186 ts_tablet_manager.cc:1397] T 6297467749854de8b8f2350156052bb2 P 6944457d5a3046a2b3623fc868ee3aa1: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260812 01:19:00.904742 27226 tablet_replica.cc:330] T 386a067d9c73409ba12f9f1a5d2d84de P 6944457d5a3046a2b3623fc868ee3aa1: stopping tablet replica
I20260812 01:19:00.904495 27228 tablet_replica.cc:330] T 386a067d9c73409ba12f9f1a5d2d84de P 4ecd5f366111458aa0761a3ed177ae2e: stopping tablet replica
I20260812 01:19:00.904903 27228 raft_consensus.cc:2229] T 386a067d9c73409ba12f9f1a5d2d84de P 4ecd5f366111458aa0761a3ed177ae2e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 01:19:00.905040 27228 raft_consensus.cc:2258] T 386a067d9c73409ba12f9f1a5d2d84de P 4ecd5f366111458aa0761a3ed177ae2e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:19:00.904714 27186 raft_consensus.cc:348] T 6297467749854de8b8f2350156052bb2 P 6944457d5a3046a2b3623fc868ee3aa1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6944457d5a3046a2b3623fc868ee3aa1" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 38473 } } peers { permanent_uuid: "4ecd5f366111458aa0761a3ed177ae2e" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 38587 } } peers { permanent_uuid: "2fc33b8fef874582803e886295a87edb" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34543 } }
I20260812 01:19:00.905179 27186 raft_consensus.cc:374] T 6297467749854de8b8f2350156052bb2 P 6944457d5a3046a2b3623fc868ee3aa1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:19:00.905210 27186 raft_consensus.cc:729] T 6297467749854de8b8f2350156052bb2 P 6944457d5a3046a2b3623fc868ee3aa1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6944457d5a3046a2b3623fc868ee3aa1, State: Initialized, Role: FOLLOWER
I20260812 01:19:00.905230 27226 raft_consensus.cc:2229] T 386a067d9c73409ba12f9f1a5d2d84de P 6944457d5a3046a2b3623fc868ee3aa1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 01:19:00.905253 27186 consensus_queue.cc:260] T 6297467749854de8b8f2350156052bb2 P 6944457d5a3046a2b3623fc868ee3aa1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6944457d5a3046a2b3623fc868ee3aa1" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 38473 } } peers { permanent_uuid: "4ecd5f366111458aa0761a3ed177ae2e" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 38587 } } peers { permanent_uuid: "2fc33b8fef874582803e886295a87edb" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34543 } }
I20260812 01:19:00.905316 27226 raft_consensus.cc:2258] T 386a067d9c73409ba12f9f1a5d2d84de P 6944457d5a3046a2b3623fc868ee3aa1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:19:00.905355 27186 ts_tablet_manager.cc:1428] T 6297467749854de8b8f2350156052bb2 P 6944457d5a3046a2b3623fc868ee3aa1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 01:19:00.905712 27228 ts_tablet_manager.cc:1905] T 386a067d9c73409ba12f9f1a5d2d84de P 4ecd5f366111458aa0761a3ed177ae2e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:19:00.905826 27189 tablet_bootstrap.cc:492] T 6297467749854de8b8f2350156052bb2 P 4ecd5f366111458aa0761a3ed177ae2e: No bootstrap required, opened a new log
I20260812 01:19:00.905896 27189 ts_tablet_manager.cc:1397] T 6297467749854de8b8f2350156052bb2 P 4ecd5f366111458aa0761a3ed177ae2e: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20260812 01:19:00.906116 27226 ts_tablet_manager.cc:1905] T 386a067d9c73409ba12f9f1a5d2d84de P 6944457d5a3046a2b3623fc868ee3aa1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:19:00.906072 27189 raft_consensus.cc:348] T 6297467749854de8b8f2350156052bb2 P 4ecd5f366111458aa0761a3ed177ae2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6944457d5a3046a2b3623fc868ee3aa1" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 38473 } } peers { permanent_uuid: "4ecd5f366111458aa0761a3ed177ae2e" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 38587 } } peers { permanent_uuid: "2fc33b8fef874582803e886295a87edb" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34543 } }
I20260812 01:19:00.906183 27189 raft_consensus.cc:374] T 6297467749854de8b8f2350156052bb2 P 4ecd5f366111458aa0761a3ed177ae2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:19:00.906213 27189 raft_consensus.cc:729] T 6297467749854de8b8f2350156052bb2 P 4ecd5f366111458aa0761a3ed177ae2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ecd5f366111458aa0761a3ed177ae2e, State: Initialized, Role: FOLLOWER
I20260812 01:19:00.906267 27189 consensus_queue.cc:260] T 6297467749854de8b8f2350156052bb2 P 4ecd5f366111458aa0761a3ed177ae2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6944457d5a3046a2b3623fc868ee3aa1" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 38473 } } peers { permanent_uuid: "4ecd5f366111458aa0761a3ed177ae2e" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 38587 } } peers { permanent_uuid: "2fc33b8fef874582803e886295a87edb" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34543 } }
I20260812 01:19:00.906378 27189 ts_tablet_manager.cc:1428] T 6297467749854de8b8f2350156052bb2 P 4ecd5f366111458aa0761a3ed177ae2e: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260812 01:19:00.907630 27228 ts_tablet_manager.cc:1918] T 386a067d9c73409ba12f9f1a5d2d84de P 4ecd5f366111458aa0761a3ed177ae2e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 01:19:00.907708 27228 log.cc:1196] T 386a067d9c73409ba12f9f1a5d2d84de P 4ecd5f366111458aa0761a3ed177ae2e: Deleting WAL directory at /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/minicluster-data/ts-1/wal/wals/386a067d9c73409ba12f9f1a5d2d84de
I20260812 01:19:00.908015 27226 ts_tablet_manager.cc:1918] T 386a067d9c73409ba12f9f1a5d2d84de P 6944457d5a3046a2b3623fc868ee3aa1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 01:19:00.908053 27228 ts_tablet_manager.cc:1939] T 386a067d9c73409ba12f9f1a5d2d84de P 4ecd5f366111458aa0761a3ed177ae2e: Deleting consensus metadata
I20260812 01:19:00.908139 27226 log.cc:1196] T 386a067d9c73409ba12f9f1a5d2d84de P 6944457d5a3046a2b3623fc868ee3aa1: Deleting WAL directory at /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/minicluster-data/ts-0/wal/wals/386a067d9c73409ba12f9f1a5d2d84de
I20260812 01:19:00.908468 27226 ts_tablet_manager.cc:1939] T 386a067d9c73409ba12f9f1a5d2d84de P 6944457d5a3046a2b3623fc868ee3aa1: Deleting consensus metadata
I20260812 01:19:00.908531 26311 catalog_manager.cc:4841] TS 4ecd5f366111458aa0761a3ed177ae2e (127.24.0.130:38587): tablet 386a067d9c73409ba12f9f1a5d2d84de (table test-table [id=858080bbcebd43759db8edaf5c7a1e83]) successfully deleted
I20260812 01:19:00.908890 26308 catalog_manager.cc:4841] TS 6944457d5a3046a2b3623fc868ee3aa1 (127.24.0.129:38473): tablet 386a067d9c73409ba12f9f1a5d2d84de (table test-table [id=858080bbcebd43759db8edaf5c7a1e83]) successfully deleted
I20260812 01:19:00.928628 27207 raft_consensus.cc:482] T 6297467749854de8b8f2350156052bb2 P 2fc33b8fef874582803e886295a87edb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 01:19:00.928738 27207 raft_consensus.cc:504] T 6297467749854de8b8f2350156052bb2 P 2fc33b8fef874582803e886295a87edb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6944457d5a3046a2b3623fc868ee3aa1" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 38473 } } peers { permanent_uuid: "4ecd5f366111458aa0761a3ed177ae2e" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 38587 } } peers { permanent_uuid: "2fc33b8fef874582803e886295a87edb" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34543 } }
I20260812 01:19:00.928915 27207 leader_election.cc:288] T 6297467749854de8b8f2350156052bb2 P 2fc33b8fef874582803e886295a87edb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6944457d5a3046a2b3623fc868ee3aa1 (127.24.0.129:38473), 4ecd5f366111458aa0761a3ed177ae2e (127.24.0.130:38587)
I20260812 01:19:00.929097 26584 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6297467749854de8b8f2350156052bb2" candidate_uuid: "2fc33b8fef874582803e886295a87edb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6944457d5a3046a2b3623fc868ee3aa1" is_pre_election: true
I20260812 01:19:00.929090 26832 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6297467749854de8b8f2350156052bb2" candidate_uuid: "2fc33b8fef874582803e886295a87edb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ecd5f366111458aa0761a3ed177ae2e" is_pre_election: true
I20260812 01:19:00.929193 26584 raft_consensus.cc:2454] T 6297467749854de8b8f2350156052bb2 P 6944457d5a3046a2b3623fc868ee3aa1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2fc33b8fef874582803e886295a87edb in term 0.
I20260812 01:19:00.929193 26832 raft_consensus.cc:2454] T 6297467749854de8b8f2350156052bb2 P 4ecd5f366111458aa0761a3ed177ae2e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2fc33b8fef874582803e886295a87edb in term 0.
I20260812 01:19:00.929389 26942 leader_election.cc:302] T 6297467749854de8b8f2350156052bb2 P 2fc33b8fef874582803e886295a87edb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2fc33b8fef874582803e886295a87edb, 4ecd5f366111458aa0761a3ed177ae2e; no voters:
I20260812 01:19:00.929477 27207 raft_consensus.cc:2781] T 6297467749854de8b8f2350156052bb2 P 2fc33b8fef874582803e886295a87edb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 01:19:00.929524 27207 raft_consensus.cc:482] T 6297467749854de8b8f2350156052bb2 P 2fc33b8fef874582803e886295a87edb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 01:19:00.929562 27207 raft_consensus.cc:3037] T 6297467749854de8b8f2350156052bb2 P 2fc33b8fef874582803e886295a87edb [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:19:00.930434 27207 raft_consensus.cc:504] T 6297467749854de8b8f2350156052bb2 P 2fc33b8fef874582803e886295a87edb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6944457d5a3046a2b3623fc868ee3aa1" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 38473 } } peers { permanent_uuid: "4ecd5f366111458aa0761a3ed177ae2e" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 38587 } } peers { permanent_uuid: "2fc33b8fef874582803e886295a87edb" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34543 } }
I20260812 01:19:00.930583 27207 leader_election.cc:288] T 6297467749854de8b8f2350156052bb2 P 2fc33b8fef874582803e886295a87edb [CANDIDATE]: Term 1 election: Requested vote from peers 6944457d5a3046a2b3623fc868ee3aa1 (127.24.0.129:38473), 4ecd5f366111458aa0761a3ed177ae2e (127.24.0.130:38587)
I20260812 01:19:00.930721 26832 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6297467749854de8b8f2350156052bb2" candidate_uuid: "2fc33b8fef874582803e886295a87edb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ecd5f366111458aa0761a3ed177ae2e"
I20260812 01:19:00.930802 26832 raft_consensus.cc:3037] T 6297467749854de8b8f2350156052bb2 P 4ecd5f366111458aa0761a3ed177ae2e [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:19:00.930768 26584 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6297467749854de8b8f2350156052bb2" candidate_uuid: "2fc33b8fef874582803e886295a87edb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6944457d5a3046a2b3623fc868ee3aa1"
I20260812 01:19:00.930833 26584 raft_consensus.cc:3037] T 6297467749854de8b8f2350156052bb2 P 6944457d5a3046a2b3623fc868ee3aa1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:19:00.931543 26584 raft_consensus.cc:2454] T 6297467749854de8b8f2350156052bb2 P 6944457d5a3046a2b3623fc868ee3aa1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2fc33b8fef874582803e886295a87edb in term 1.
I20260812 01:19:00.931543 26832 raft_consensus.cc:2454] T 6297467749854de8b8f2350156052bb2 P 4ecd5f366111458aa0761a3ed177ae2e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2fc33b8fef874582803e886295a87edb in term 1.
I20260812 01:19:00.931691 26942 leader_election.cc:302] T 6297467749854de8b8f2350156052bb2 P 2fc33b8fef874582803e886295a87edb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2fc33b8fef874582803e886295a87edb, 4ecd5f366111458aa0761a3ed177ae2e; no voters:
I20260812 01:19:00.931811 27207 raft_consensus.cc:2781] T 6297467749854de8b8f2350156052bb2 P 2fc33b8fef874582803e886295a87edb [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:19:00.931865 27207 raft_consensus.cc:686] T 6297467749854de8b8f2350156052bb2 P 2fc33b8fef874582803e886295a87edb [term 1 LEADER]: Becoming Leader. State: Replica: 2fc33b8fef874582803e886295a87edb, State: Running, Role: LEADER
I20260812 01:19:00.931910 27207 consensus_queue.cc:237] T 6297467749854de8b8f2350156052bb2 P 2fc33b8fef874582803e886295a87edb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6944457d5a3046a2b3623fc868ee3aa1" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 38473 } } peers { permanent_uuid: "4ecd5f366111458aa0761a3ed177ae2e" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 38587 } } peers { permanent_uuid: "2fc33b8fef874582803e886295a87edb" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34543 } }
I20260812 01:19:00.932451 26334 catalog_manager.cc:5495] T 6297467749854de8b8f2350156052bb2 P 2fc33b8fef874582803e886295a87edb reported cstate change: term changed from 0 to 1, leader changed from <none> to 2fc33b8fef874582803e886295a87edb (127.24.0.131). New cstate: current_term: 1 leader_uuid: "2fc33b8fef874582803e886295a87edb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6944457d5a3046a2b3623fc868ee3aa1" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 38473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4ecd5f366111458aa0761a3ed177ae2e" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 38587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2fc33b8fef874582803e886295a87edb" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34543 } health_report { overall_health: HEALTHY } } }
Aug 12 01:19:00 dist-test-slave-t3q3 krb5kdc[26269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497540, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.190@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
I20260812 01:19:00.961177 26415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:00.952894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59894 (local address 127.24.0.190:42265)
0812 01:19:00.952958 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:00.952960 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:00.953104 (+ 144us) server_negotiation.cc:407] Connection header received
0812 01:19:00.953193 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:00.953194 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:00.953213 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:00.953277 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:00.953903 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:00.954410 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:00.955121 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:00.955303 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:00.955390 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:00.958231 (+ 2841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:00.958232 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:00.958232 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:00.958239 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:00.959710 (+ 1471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:00.960151 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:00.960152 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:00.960153 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:00.960214 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:00.960417 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:00.960419 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:00.960419 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:00.960538 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:00.960724 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:00.960904 (+ 180us) server_negotiation.cc:299] Negotiation successful
0812 01:19:00.961093 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":113,"server-negotiator.queue_time_us":37}
I20260812 01:19:00.961189 27244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:00.952824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:42265 (local address 127.0.0.1:59894)
0812 01:19:00.953020 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:00.953036 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:00.953120 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:00.953281 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:00.953282 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:00.953728 (+ 446us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:00.953781 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:00.953785 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:00.954499 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:00.954500 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:00.955024 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:00.955025 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:00.955391 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:00.955392 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:00.955443 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:00.956126 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:00.956131 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:00.957996 (+ 1865us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:19:00.959817 (+ 1821us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:00.959826 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:00.959828 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:00.960032 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:00.960281 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:00.960282 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:00.960282 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:00.960352 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:00.960714 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:19:00.960722 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:19:00.960780 (+ 58us) client_negotiation.cc:769] Sending connection context
0812 01:19:00.960878 (+ 98us) client_negotiation.cc:240] Negotiation successful
0812 01:19:00.961039 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"mutex_wait_us":2,"thread_start_us":92,"threads_started":1}
I20260812 01:19:00.964128 26334 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.24.0.190@KRBTEST.COM'} at 127.0.0.1:59894:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260812 01:19:00.964411 26334 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 01:19:00.965929 27004 tablet_service.cc:1467] Processing CreateTablet for tablet b80968488a454a6f9e4f74ab74370df0 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f28217e609b94ca2bc47a07add2ba786]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 01:19:00.965924 26552 tablet_service.cc:1467] Processing CreateTablet for tablet b80968488a454a6f9e4f74ab74370df0 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f28217e609b94ca2bc47a07add2ba786]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 01:19:00.965919 26795 tablet_service.cc:1467] Processing CreateTablet for tablet b80968488a454a6f9e4f74ab74370df0 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f28217e609b94ca2bc47a07add2ba786]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 01:19:00.966107 26552 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b80968488a454a6f9e4f74ab74370df0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:19:00.966107 26795 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b80968488a454a6f9e4f74ab74370df0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:19:00.966109 27004 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b80968488a454a6f9e4f74ab74370df0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:19:00.969089 27185 tablet_bootstrap.cc:492] T b80968488a454a6f9e4f74ab74370df0 P 2fc33b8fef874582803e886295a87edb: Bootstrap starting.
I20260812 01:19:00.969129 27186 tablet_bootstrap.cc:492] T b80968488a454a6f9e4f74ab74370df0 P 6944457d5a3046a2b3623fc868ee3aa1: Bootstrap starting.
I20260812 01:19:00.969129 27189 tablet_bootstrap.cc:492] T b80968488a454a6f9e4f74ab74370df0 P 4ecd5f366111458aa0761a3ed177ae2e: Bootstrap starting.
I20260812 01:19:00.969478 27185 tablet_bootstrap.cc:654] T b80968488a454a6f9e4f74ab74370df0 P 2fc33b8fef874582803e886295a87edb: Neither blocks nor log segments found. Creating new log.
I20260812 01:19:00.969583 27186 tablet_bootstrap.cc:654] T b80968488a454a6f9e4f74ab74370df0 P 6944457d5a3046a2b3623fc868ee3aa1: Neither blocks nor log segments found. Creating new log.
I20260812 01:19:00.969594 27189 tablet_bootstrap.cc:654] T b80968488a454a6f9e4f74ab74370df0 P 4ecd5f366111458aa0761a3ed177ae2e: Neither blocks nor log segments found. Creating new log.
I20260812 01:19:00.970366 27189 tablet_bootstrap.cc:492] T b80968488a454a6f9e4f74ab74370df0 P 4ecd5f366111458aa0761a3ed177ae2e: No bootstrap required, opened a new log
I20260812 01:19:00.970417 27189 ts_tablet_manager.cc:1397] T b80968488a454a6f9e4f74ab74370df0 P 4ecd5f366111458aa0761a3ed177ae2e: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260812 01:19:00.970438 27185 tablet_bootstrap.cc:492] T b80968488a454a6f9e4f74ab74370df0 P 2fc33b8fef874582803e886295a87edb: No bootstrap required, opened a new log
I20260812 01:19:00.970482 27185 ts_tablet_manager.cc:1397] T b80968488a454a6f9e4f74ab74370df0 P 2fc33b8fef874582803e886295a87edb: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260812 01:19:00.970563 27189 raft_consensus.cc:348] T b80968488a454a6f9e4f74ab74370df0 P 4ecd5f366111458aa0761a3ed177ae2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6944457d5a3046a2b3623fc868ee3aa1" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 38473 } } peers { permanent_uuid: "4ecd5f366111458aa0761a3ed177ae2e" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 38587 } } peers { permanent_uuid: "2fc33b8fef874582803e886295a87edb" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34543 } }
I20260812 01:19:00.970613 27189 raft_consensus.cc:374] T b80968488a454a6f9e4f74ab74370df0 P 4ecd5f366111458aa0761a3ed177ae2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:19:00.970626 27189 raft_consensus.cc:729] T b80968488a454a6f9e4f74ab74370df0 P 4ecd5f366111458aa0761a3ed177ae2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ecd5f366111458aa0761a3ed177ae2e, State: Initialized, Role: FOLLOWER
I20260812 01:19:00.970628 27185 raft_consensus.cc:348] T b80968488a454a6f9e4f74ab74370df0 P 2fc33b8fef874582803e886295a87edb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6944457d5a3046a2b3623fc868ee3aa1" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 38473 } } peers { permanent_uuid: "4ecd5f366111458aa0761a3ed177ae2e" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 38587 } } peers { permanent_uuid: "2fc33b8fef874582803e886295a87edb" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34543 } }
I20260812 01:19:00.970703 27185 raft_consensus.cc:374] T b80968488a454a6f9e4f74ab74370df0 P 2fc33b8fef874582803e886295a87edb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:19:00.970686 27189 consensus_queue.cc:260] T b80968488a454a6f9e4f74ab74370df0 P 4ecd5f366111458aa0761a3ed177ae2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6944457d5a3046a2b3623fc868ee3aa1" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 38473 } } peers { permanent_uuid: "4ecd5f366111458aa0761a3ed177ae2e" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 38587 } } peers { permanent_uuid: "2fc33b8fef874582803e886295a87edb" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34543 } }
I20260812 01:19:00.970724 27185 raft_consensus.cc:729] T b80968488a454a6f9e4f74ab74370df0 P 2fc33b8fef874582803e886295a87edb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fc33b8fef874582803e886295a87edb, State: Initialized, Role: FOLLOWER
I20260812 01:19:00.970779 27189 ts_tablet_manager.cc:1428] T b80968488a454a6f9e4f74ab74370df0 P 4ecd5f366111458aa0761a3ed177ae2e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:00.970768 27185 consensus_queue.cc:260] T b80968488a454a6f9e4f74ab74370df0 P 2fc33b8fef874582803e886295a87edb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6944457d5a3046a2b3623fc868ee3aa1" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 38473 } } peers { permanent_uuid: "4ecd5f366111458aa0761a3ed177ae2e" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 38587 } } peers { permanent_uuid: "2fc33b8fef874582803e886295a87edb" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34543 } }
I20260812 01:19:00.970849 27185 ts_tablet_manager.cc:1428] T b80968488a454a6f9e4f74ab74370df0 P 2fc33b8fef874582803e886295a87edb: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:00.970835 27203 tablet_replica.cc:439] T b80968488a454a6f9e4f74ab74370df0 P 4ecd5f366111458aa0761a3ed177ae2e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6944457d5a3046a2b3623fc868ee3aa1" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 38473 } } peers { permanent_uuid: "4ecd5f366111458aa0761a3ed177ae2e" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 38587 } } peers { permanent_uuid: "2fc33b8fef874582803e886295a87edb" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34543 } } }
I20260812 01:19:00.970921 27203 tablet_replica.cc:442] T b80968488a454a6f9e4f74ab74370df0 P 4ecd5f366111458aa0761a3ed177ae2e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 01:19:00.971016 27207 tablet_replica.cc:439] T b80968488a454a6f9e4f74ab74370df0 P 2fc33b8fef874582803e886295a87edb: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6944457d5a3046a2b3623fc868ee3aa1" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 38473 } } peers { permanent_uuid: "4ecd5f366111458aa0761a3ed177ae2e" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 38587 } } peers { permanent_uuid: "2fc33b8fef874582803e886295a87edb" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34543 } } }
I20260812 01:19:00.971083 27207 tablet_replica.cc:442] T b80968488a454a6f9e4f74ab74370df0 P 2fc33b8fef874582803e886295a87edb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 01:19:00.971223 27186 tablet_bootstrap.cc:492] T b80968488a454a6f9e4f74ab74370df0 P 6944457d5a3046a2b3623fc868ee3aa1: No bootstrap required, opened a new log
I20260812 01:19:00.971269 27186 ts_tablet_manager.cc:1397] T b80968488a454a6f9e4f74ab74370df0 P 6944457d5a3046a2b3623fc868ee3aa1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 01:19:00.971421 27186 raft_consensus.cc:348] T b80968488a454a6f9e4f74ab74370df0 P 6944457d5a3046a2b3623fc868ee3aa1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6944457d5a3046a2b3623fc868ee3aa1" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 38473 } } peers { permanent_uuid: "4ecd5f366111458aa0761a3ed177ae2e" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 38587 } } peers { permanent_uuid: "2fc33b8fef874582803e886295a87edb" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34543 } }
I20260812 01:19:00.971499 27186 raft_consensus.cc:374] T b80968488a454a6f9e4f74ab74370df0 P 6944457d5a3046a2b3623fc868ee3aa1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:19:00.971534 27186 raft_consensus.cc:729] T b80968488a454a6f9e4f74ab74370df0 P 6944457d5a3046a2b3623fc868ee3aa1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6944457d5a3046a2b3623fc868ee3aa1, State: Initialized, Role: FOLLOWER
I20260812 01:19:00.971578 27186 consensus_queue.cc:260] T b80968488a454a6f9e4f74ab74370df0 P 6944457d5a3046a2b3623fc868ee3aa1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6944457d5a3046a2b3623fc868ee3aa1" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 38473 } } peers { permanent_uuid: "4ecd5f366111458aa0761a3ed177ae2e" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 38587 } } peers { permanent_uuid: "2fc33b8fef874582803e886295a87edb" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34543 } }
I20260812 01:19:00.971674 27186 ts_tablet_manager.cc:1428] T b80968488a454a6f9e4f74ab74370df0 P 6944457d5a3046a2b3623fc868ee3aa1: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:00.971733 27200 tablet_replica.cc:439] T b80968488a454a6f9e4f74ab74370df0 P 6944457d5a3046a2b3623fc868ee3aa1: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6944457d5a3046a2b3623fc868ee3aa1" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 38473 } } peers { permanent_uuid: "4ecd5f366111458aa0761a3ed177ae2e" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 38587 } } peers { permanent_uuid: "2fc33b8fef874582803e886295a87edb" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34543 } } }
I20260812 01:19:00.971802 27200 tablet_replica.cc:442] T b80968488a454a6f9e4f74ab74370df0 P 6944457d5a3046a2b3623fc868ee3aa1: This TxnStatusTablet replica's current role is: FOLLOWER
W20260812 01:19:00.973682 26657 tablet.cc:2369] T 6297467749854de8b8f2350156052bb2 P 6944457d5a3046a2b3623fc868ee3aa1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 01:19:01.054283 27148 tablet.cc:2369] T 6297467749854de8b8f2350156052bb2 P 2fc33b8fef874582803e886295a87edb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 01:19:01.174647 27207 raft_consensus.cc:482] T b80968488a454a6f9e4f74ab74370df0 P 2fc33b8fef874582803e886295a87edb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 01:19:01.174757 27207 raft_consensus.cc:504] T b80968488a454a6f9e4f74ab74370df0 P 2fc33b8fef874582803e886295a87edb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6944457d5a3046a2b3623fc868ee3aa1" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 38473 } } peers { permanent_uuid: "4ecd5f366111458aa0761a3ed177ae2e" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 38587 } } peers { permanent_uuid: "2fc33b8fef874582803e886295a87edb" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34543 } }
I20260812 01:19:01.174948 27207 leader_election.cc:288] T b80968488a454a6f9e4f74ab74370df0 P 2fc33b8fef874582803e886295a87edb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6944457d5a3046a2b3623fc868ee3aa1 (127.24.0.129:38473), 4ecd5f366111458aa0761a3ed177ae2e (127.24.0.130:38587)
I20260812 01:19:01.175233 26832 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b80968488a454a6f9e4f74ab74370df0" candidate_uuid: "2fc33b8fef874582803e886295a87edb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ecd5f366111458aa0761a3ed177ae2e" is_pre_election: true
I20260812 01:19:01.175242 26584 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b80968488a454a6f9e4f74ab74370df0" candidate_uuid: "2fc33b8fef874582803e886295a87edb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6944457d5a3046a2b3623fc868ee3aa1" is_pre_election: true
I20260812 01:19:01.175351 26832 raft_consensus.cc:2454] T b80968488a454a6f9e4f74ab74370df0 P 4ecd5f366111458aa0761a3ed177ae2e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2fc33b8fef874582803e886295a87edb in term 0.
I20260812 01:19:01.175347 26584 raft_consensus.cc:2454] T b80968488a454a6f9e4f74ab74370df0 P 6944457d5a3046a2b3623fc868ee3aa1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2fc33b8fef874582803e886295a87edb in term 0.
I20260812 01:19:01.175529 26942 leader_election.cc:302] T b80968488a454a6f9e4f74ab74370df0 P 2fc33b8fef874582803e886295a87edb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2fc33b8fef874582803e886295a87edb, 4ecd5f366111458aa0761a3ed177ae2e; no voters:
I20260812 01:19:01.175654 27207 raft_consensus.cc:2781] T b80968488a454a6f9e4f74ab74370df0 P 2fc33b8fef874582803e886295a87edb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 01:19:01.175699 27207 raft_consensus.cc:482] T b80968488a454a6f9e4f74ab74370df0 P 2fc33b8fef874582803e886295a87edb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 01:19:01.175714 27207 raft_consensus.cc:3037] T b80968488a454a6f9e4f74ab74370df0 P 2fc33b8fef874582803e886295a87edb [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:19:01.176502 27207 raft_consensus.cc:504] T b80968488a454a6f9e4f74ab74370df0 P 2fc33b8fef874582803e886295a87edb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6944457d5a3046a2b3623fc868ee3aa1" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 38473 } } peers { permanent_uuid: "4ecd5f366111458aa0761a3ed177ae2e" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 38587 } } peers { permanent_uuid: "2fc33b8fef874582803e886295a87edb" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34543 } }
I20260812 01:19:01.176641 27207 leader_election.cc:288] T b80968488a454a6f9e4f74ab74370df0 P 2fc33b8fef874582803e886295a87edb [CANDIDATE]: Term 1 election: Requested vote from peers 6944457d5a3046a2b3623fc868ee3aa1 (127.24.0.129:38473), 4ecd5f366111458aa0761a3ed177ae2e (127.24.0.130:38587)
I20260812 01:19:01.176829 26832 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b80968488a454a6f9e4f74ab74370df0" candidate_uuid: "2fc33b8fef874582803e886295a87edb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ecd5f366111458aa0761a3ed177ae2e"
I20260812 01:19:01.176836 26584 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b80968488a454a6f9e4f74ab74370df0" candidate_uuid: "2fc33b8fef874582803e886295a87edb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6944457d5a3046a2b3623fc868ee3aa1"
I20260812 01:19:01.176930 26832 raft_consensus.cc:3037] T b80968488a454a6f9e4f74ab74370df0 P 4ecd5f366111458aa0761a3ed177ae2e [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:19:01.176929 26584 raft_consensus.cc:3037] T b80968488a454a6f9e4f74ab74370df0 P 6944457d5a3046a2b3623fc868ee3aa1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:19:01.177541 26584 raft_consensus.cc:2454] T b80968488a454a6f9e4f74ab74370df0 P 6944457d5a3046a2b3623fc868ee3aa1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2fc33b8fef874582803e886295a87edb in term 1.
I20260812 01:19:01.177620 26832 raft_consensus.cc:2454] T b80968488a454a6f9e4f74ab74370df0 P 4ecd5f366111458aa0761a3ed177ae2e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2fc33b8fef874582803e886295a87edb in term 1.
I20260812 01:19:01.177769 26940 leader_election.cc:302] T b80968488a454a6f9e4f74ab74370df0 P 2fc33b8fef874582803e886295a87edb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2fc33b8fef874582803e886295a87edb, 6944457d5a3046a2b3623fc868ee3aa1; no voters:
I20260812 01:19:01.177920 27207 raft_consensus.cc:2781] T b80968488a454a6f9e4f74ab74370df0 P 2fc33b8fef874582803e886295a87edb [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:19:01.177983 27207 raft_consensus.cc:686] T b80968488a454a6f9e4f74ab74370df0 P 2fc33b8fef874582803e886295a87edb [term 1 LEADER]: Becoming Leader. State: Replica: 2fc33b8fef874582803e886295a87edb, State: Running, Role: LEADER
I20260812 01:19:01.178049 27207 consensus_queue.cc:237] T b80968488a454a6f9e4f74ab74370df0 P 2fc33b8fef874582803e886295a87edb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6944457d5a3046a2b3623fc868ee3aa1" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 38473 } } peers { permanent_uuid: "4ecd5f366111458aa0761a3ed177ae2e" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 38587 } } peers { permanent_uuid: "2fc33b8fef874582803e886295a87edb" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34543 } }
I20260812 01:19:01.178333 27199 tablet_replica.cc:439] T b80968488a454a6f9e4f74ab74370df0 P 2fc33b8fef874582803e886295a87edb: TxnStatusTablet state changed. Reason: New leader 2fc33b8fef874582803e886295a87edb. Latest consensus state: current_term: 1 leader_uuid: "2fc33b8fef874582803e886295a87edb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6944457d5a3046a2b3623fc868ee3aa1" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 38473 } } peers { permanent_uuid: "4ecd5f366111458aa0761a3ed177ae2e" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 38587 } } peers { permanent_uuid: "2fc33b8fef874582803e886295a87edb" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34543 } } }
I20260812 01:19:01.178423 27199 tablet_replica.cc:442] T b80968488a454a6f9e4f74ab74370df0 P 2fc33b8fef874582803e886295a87edb: This TxnStatusTablet replica's current role is: LEADER
I20260812 01:19:01.178676 26333 catalog_manager.cc:5495] T b80968488a454a6f9e4f74ab74370df0 P 2fc33b8fef874582803e886295a87edb reported cstate change: term changed from 0 to 1, leader changed from <none> to 2fc33b8fef874582803e886295a87edb (127.24.0.131). New cstate: current_term: 1 leader_uuid: "2fc33b8fef874582803e886295a87edb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6944457d5a3046a2b3623fc868ee3aa1" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 38473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4ecd5f366111458aa0761a3ed177ae2e" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 38587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2fc33b8fef874582803e886295a87edb" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34543 } health_report { overall_health: HEALTHY } } }
I20260812 01:19:01.179005 27252 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260812 01:19:01.179145 27252 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260812 01:19:01.199261 27244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:01.196627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.131:34543 (local address 127.0.0.1:38270)
0812 01:19:01.196713 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:01.196727 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:01.196787 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:01.196932 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:01.196933 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:01.196934 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 01:19:01.196982 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:01.196984 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:01.197856 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:01.197857 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:01.198419 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:01.198420 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:01.198876 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:01.198877 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:01.198924 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:01.198939 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 01:19:01.199112 (+ 173us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 01:19:01.199119 (+ 7us) client_negotiation.cc:769] Sending connection context
0812 01:19:01.199154 (+ 35us) client_negotiation.cc:240] Negotiation successful
0812 01:19:01.199201 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260812 01:19:01.199258 27184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:01.196712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38270 (local address 127.24.0.131:34543)
0812 01:19:01.196765 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:01.196767 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:01.196778 (+ 11us) server_negotiation.cc:407] Connection header received
0812 01:19:01.196845 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:01.196846 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:01.196857 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:01.196915 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:19:01.197044 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:01.197785 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:01.198497 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:01.198798 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:01.198871 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:01.199017 (+ 146us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:19:01.199051 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:19:01.199105 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:01.199157 (+ 52us) server_negotiation.cc:299] Negotiation successful
0812 01:19:01.199179 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 01:19:01.200424 26832 raft_consensus.cc:1261] T b80968488a454a6f9e4f74ab74370df0 P 4ecd5f366111458aa0761a3ed177ae2e [term 1 FOLLOWER]: Refusing update from remote peer 2fc33b8fef874582803e886295a87edb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:19:01.200474 26584 raft_consensus.cc:1261] T b80968488a454a6f9e4f74ab74370df0 P 6944457d5a3046a2b3623fc868ee3aa1 [term 1 FOLLOWER]: Refusing update from remote peer 2fc33b8fef874582803e886295a87edb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:19:01.200641 27199 consensus_queue.cc:1035] T b80968488a454a6f9e4f74ab74370df0 P 2fc33b8fef874582803e886295a87edb [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ecd5f366111458aa0761a3ed177ae2e" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 38587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:19:01.200748 27207 consensus_queue.cc:1035] T b80968488a454a6f9e4f74ab74370df0 P 2fc33b8fef874582803e886295a87edb [LEADER]: Connected to new peer: Peer: permanent_uuid: "6944457d5a3046a2b3623fc868ee3aa1" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 38473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:19:01.201259 27203 tablet_replica.cc:439] T b80968488a454a6f9e4f74ab74370df0 P 4ecd5f366111458aa0761a3ed177ae2e: TxnStatusTablet state changed. Reason: New leader 2fc33b8fef874582803e886295a87edb. Latest consensus state: current_term: 1 leader_uuid: "2fc33b8fef874582803e886295a87edb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6944457d5a3046a2b3623fc868ee3aa1" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 38473 } } peers { permanent_uuid: "4ecd5f366111458aa0761a3ed177ae2e" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 38587 } } peers { permanent_uuid: "2fc33b8fef874582803e886295a87edb" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34543 } } }
I20260812 01:19:01.201263 27200 tablet_replica.cc:439] T b80968488a454a6f9e4f74ab74370df0 P 6944457d5a3046a2b3623fc868ee3aa1: TxnStatusTablet state changed. Reason: New leader 2fc33b8fef874582803e886295a87edb. Latest consensus state: current_term: 1 leader_uuid: "2fc33b8fef874582803e886295a87edb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6944457d5a3046a2b3623fc868ee3aa1" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 38473 } } peers { permanent_uuid: "4ecd5f366111458aa0761a3ed177ae2e" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 38587 } } peers { permanent_uuid: "2fc33b8fef874582803e886295a87edb" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34543 } } }
I20260812 01:19:01.201359 27203 tablet_replica.cc:442] T b80968488a454a6f9e4f74ab74370df0 P 4ecd5f366111458aa0761a3ed177ae2e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 01:19:01.201371 27200 tablet_replica.cc:442] T b80968488a454a6f9e4f74ab74370df0 P 6944457d5a3046a2b3623fc868ee3aa1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 01:19:01.202350 27220 tablet_replica.cc:439] T b80968488a454a6f9e4f74ab74370df0 P 2fc33b8fef874582803e886295a87edb: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2fc33b8fef874582803e886295a87edb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6944457d5a3046a2b3623fc868ee3aa1" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 38473 } } peers { permanent_uuid: "4ecd5f366111458aa0761a3ed177ae2e" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 38587 } } peers { permanent_uuid: "2fc33b8fef874582803e886295a87edb" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34543 } } }
I20260812 01:19:01.202445 27220 tablet_replica.cc:442] T b80968488a454a6f9e4f74ab74370df0 P 2fc33b8fef874582803e886295a87edb: This TxnStatusTablet replica's current role is: LEADER
I20260812 01:19:01.202468 27203 tablet_replica.cc:439] T b80968488a454a6f9e4f74ab74370df0 P 4ecd5f366111458aa0761a3ed177ae2e: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2fc33b8fef874582803e886295a87edb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6944457d5a3046a2b3623fc868ee3aa1" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 38473 } } peers { permanent_uuid: "4ecd5f366111458aa0761a3ed177ae2e" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 38587 } } peers { permanent_uuid: "2fc33b8fef874582803e886295a87edb" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34543 } } }
I20260812 01:19:01.202508 27220 tablet_replica.cc:439] T b80968488a454a6f9e4f74ab74370df0 P 2fc33b8fef874582803e886295a87edb: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2fc33b8fef874582803e886295a87edb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6944457d5a3046a2b3623fc868ee3aa1" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 38473 } } peers { permanent_uuid: "4ecd5f366111458aa0761a3ed177ae2e" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 38587 } } peers { permanent_uuid: "2fc33b8fef874582803e886295a87edb" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34543 } } }
I20260812 01:19:01.202554 27220 tablet_replica.cc:442] T b80968488a454a6f9e4f74ab74370df0 P 2fc33b8fef874582803e886295a87edb: This TxnStatusTablet replica's current role is: LEADER
I20260812 01:19:01.202550 27203 tablet_replica.cc:442] T b80968488a454a6f9e4f74ab74370df0 P 4ecd5f366111458aa0761a3ed177ae2e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 01:19:01.203595 27200 tablet_replica.cc:439] T b80968488a454a6f9e4f74ab74370df0 P 6944457d5a3046a2b3623fc868ee3aa1: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2fc33b8fef874582803e886295a87edb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6944457d5a3046a2b3623fc868ee3aa1" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 38473 } } peers { permanent_uuid: "4ecd5f366111458aa0761a3ed177ae2e" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 38587 } } peers { permanent_uuid: "2fc33b8fef874582803e886295a87edb" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34543 } } }
I20260812 01:19:01.203685 27200 tablet_replica.cc:442] T b80968488a454a6f9e4f74ab74370df0 P 6944457d5a3046a2b3623fc868ee3aa1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 01:19:01.209322 27159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:01.206660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.131:34543 (local address 127.24.0.131:34435)
0812 01:19:01.206748 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:01.206757 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:01.206798 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:01.206915 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:01.206916 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:01.206917 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 01:19:01.206954 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:01.206955 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:01.207825 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:01.207826 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:01.208479 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:01.208481 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:01.208955 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:01.208956 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:01.208998 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:01.209011 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 01:19:01.209179 (+ 168us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 01:19:01.209180 (+ 1us) client_negotiation.cc:769] Sending connection context
0812 01:19:01.209210 (+ 30us) client_negotiation.cc:240] Negotiation successful
0812 01:19:01.209242 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260812 01:19:01.209323 27184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:01.206723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.131:34435 (local address 127.24.0.131:34543)
0812 01:19:01.206763 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:01.206765 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:01.206791 (+ 26us) server_negotiation.cc:407] Connection header received
0812 01:19:01.206838 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:01.206838 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:01.206848 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:01.206914 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:19:01.206988 (+ 74us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:01.207725 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:01.208557 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:01.208854 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:01.208955 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:01.209088 (+ 133us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:19:01.209119 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:19:01.209180 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:01.209213 (+ 33us) server_negotiation.cc:299] Negotiation successful
0812 01:19:01.209232 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 01:19:01.211470 26584 raft_consensus.cc:1261] T 6297467749854de8b8f2350156052bb2 P 6944457d5a3046a2b3623fc868ee3aa1 [term 1 FOLLOWER]: Refusing update from remote peer 2fc33b8fef874582803e886295a87edb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:19:01.211469 26832 raft_consensus.cc:1261] T 6297467749854de8b8f2350156052bb2 P 4ecd5f366111458aa0761a3ed177ae2e [term 1 FOLLOWER]: Refusing update from remote peer 2fc33b8fef874582803e886295a87edb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:19:01.211771 27220 consensus_queue.cc:1035] T 6297467749854de8b8f2350156052bb2 P 2fc33b8fef874582803e886295a87edb [LEADER]: Connected to new peer: Peer: permanent_uuid: "6944457d5a3046a2b3623fc868ee3aa1" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 38473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:19:01.211871 27207 consensus_queue.cc:1035] T 6297467749854de8b8f2350156052bb2 P 2fc33b8fef874582803e886295a87edb [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ecd5f366111458aa0761a3ed177ae2e" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 38587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:19:01.223133 26333 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59884:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:19:01.223217 26333 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59884:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:19:01.223765 26333 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dec86a9c3f9845a383f2cc0f2a31b340: Sending DeleteTablet for 3 replicas of tablet 6297467749854de8b8f2350156052bb2
I20260812 01:19:01.224010 26795 tablet_service.cc:1514] Processing DeleteTablet for tablet 6297467749854de8b8f2350156052bb2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:19:01 UTC) from {username='kudu', principal='kudu/127.24.0.190@KRBTEST.COM'} at 127.0.0.1:48922
I20260812 01:19:01.224086 26552 tablet_service.cc:1514] Processing DeleteTablet for tablet 6297467749854de8b8f2350156052bb2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:19:01 UTC) from {username='kudu', principal='kudu/127.24.0.190@KRBTEST.COM'} at 127.0.0.1:50958
I20260812 01:19:01.224146 27228 tablet_replica.cc:330] T 6297467749854de8b8f2350156052bb2 P 4ecd5f366111458aa0761a3ed177ae2e: stopping tablet replica
I20260812 01:19:01.224133 27004 tablet_service.cc:1514] Processing DeleteTablet for tablet 6297467749854de8b8f2350156052bb2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:19:01 UTC) from {username='kudu', principal='kudu/127.24.0.190@KRBTEST.COM'} at 127.0.0.1:38244
I20260812 01:19:01.224189 27228 raft_consensus.cc:2229] T 6297467749854de8b8f2350156052bb2 P 4ecd5f366111458aa0761a3ed177ae2e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 01:19:01.224211 27229 tablet_replica.cc:330] T 6297467749854de8b8f2350156052bb2 P 2fc33b8fef874582803e886295a87edb: stopping tablet replica
I20260812 01:19:01.224231 27228 raft_consensus.cc:2258] T 6297467749854de8b8f2350156052bb2 P 4ecd5f366111458aa0761a3ed177ae2e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:19:01.224256 27229 raft_consensus.cc:2229] T 6297467749854de8b8f2350156052bb2 P 2fc33b8fef874582803e886295a87edb [term 1 LEADER]: Raft consensus shutting down.
I20260812 01:19:01.224295 27226 tablet_replica.cc:330] T 6297467749854de8b8f2350156052bb2 P 6944457d5a3046a2b3623fc868ee3aa1: stopping tablet replica
I20260812 01:19:01.224313 27229 raft_consensus.cc:2258] T 6297467749854de8b8f2350156052bb2 P 2fc33b8fef874582803e886295a87edb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:19:01.224339 27226 raft_consensus.cc:2229] T 6297467749854de8b8f2350156052bb2 P 6944457d5a3046a2b3623fc868ee3aa1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 01:19:01.224373 27226 raft_consensus.cc:2258] T 6297467749854de8b8f2350156052bb2 P 6944457d5a3046a2b3623fc868ee3aa1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:19:01.224572 27229 ts_tablet_manager.cc:1905] T 6297467749854de8b8f2350156052bb2 P 2fc33b8fef874582803e886295a87edb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:19:01.224718 27226 ts_tablet_manager.cc:1905] T 6297467749854de8b8f2350156052bb2 P 6944457d5a3046a2b3623fc868ee3aa1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:19:01.226287 27229 ts_tablet_manager.cc:1918] T 6297467749854de8b8f2350156052bb2 P 2fc33b8fef874582803e886295a87edb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 01:19:01.226357 27229 log.cc:1196] T 6297467749854de8b8f2350156052bb2 P 2fc33b8fef874582803e886295a87edb: Deleting WAL directory at /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/minicluster-data/ts-2/wal/wals/6297467749854de8b8f2350156052bb2
I20260812 01:19:01.226418 27226 ts_tablet_manager.cc:1918] T 6297467749854de8b8f2350156052bb2 P 6944457d5a3046a2b3623fc868ee3aa1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 01:19:01.226483 27226 log.cc:1196] T 6297467749854de8b8f2350156052bb2 P 6944457d5a3046a2b3623fc868ee3aa1: Deleting WAL directory at /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/minicluster-data/ts-0/wal/wals/6297467749854de8b8f2350156052bb2
I20260812 01:19:01.226572 27229 ts_tablet_manager.cc:1939] T 6297467749854de8b8f2350156052bb2 P 2fc33b8fef874582803e886295a87edb: Deleting consensus metadata
I20260812 01:19:01.226728 27226 ts_tablet_manager.cc:1939] T 6297467749854de8b8f2350156052bb2 P 6944457d5a3046a2b3623fc868ee3aa1: Deleting consensus metadata
I20260812 01:19:01.226884 26308 catalog_manager.cc:4841] TS 2fc33b8fef874582803e886295a87edb (127.24.0.131:34543): tablet 6297467749854de8b8f2350156052bb2 (table test-table [id=8a26d5ec9ce64104bc0deb7be207aad8]) successfully deleted
I20260812 01:19:01.227003 26308 catalog_manager.cc:4841] TS 6944457d5a3046a2b3623fc868ee3aa1 (127.24.0.129:38473): tablet 6297467749854de8b8f2350156052bb2 (table test-table [id=8a26d5ec9ce64104bc0deb7be207aad8]) successfully deleted
I20260812 01:19:01.227155 27228 ts_tablet_manager.cc:1905] T 6297467749854de8b8f2350156052bb2 P 4ecd5f366111458aa0761a3ed177ae2e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:19:01.228281 27228 ts_tablet_manager.cc:1918] T 6297467749854de8b8f2350156052bb2 P 4ecd5f366111458aa0761a3ed177ae2e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 01:19:01.228329 27228 log.cc:1196] T 6297467749854de8b8f2350156052bb2 P 4ecd5f366111458aa0761a3ed177ae2e: Deleting WAL directory at /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786497534195564-24578-0/minicluster-data/ts-1/wal/wals/6297467749854de8b8f2350156052bb2
I20260812 01:19:01.228480 27228 ts_tablet_manager.cc:1939] T 6297467749854de8b8f2350156052bb2 P 4ecd5f366111458aa0761a3ed177ae2e: Deleting consensus metadata
I20260812 01:19:01.228845 26311 catalog_manager.cc:4841] TS 4ecd5f366111458aa0761a3ed177ae2e (127.24.0.130:38587): tablet 6297467749854de8b8f2350156052bb2 (table test-table [id=8a26d5ec9ce64104bc0deb7be207aad8]) successfully deleted
Aug 12 01:19:01 dist-test-slave-t3q3 krb5kdc[26269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497540, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.24.0.129@KRBTEST.COM
I20260812 01:19:01.235152 27180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:01.224742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50962 (local address 127.24.0.129:38473)
0812 01:19:01.224787 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:01.224790 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:01.225081 (+ 291us) server_negotiation.cc:407] Connection header received
0812 01:19:01.225130 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:01.225131 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:01.225142 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:01.225202 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:01.225939 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:01.226717 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:01.227986 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:01.228318 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:01.228363 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:01.232032 (+ 3669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:01.232041 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:01.232041 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:01.232049 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:01.233913 (+ 1864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:01.234322 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:01.234325 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:01.234326 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:01.234394 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:01.234612 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:01.234613 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:01.234614 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:01.234787 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:01.234931 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:01.234981 (+ 50us) server_negotiation.cc:299] Negotiation successful
0812 01:19:01.235060 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
W20260812 01:19:01.235450 26646 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50962
I20260812 01:19:01.242640 26415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:01.236456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59896 (local address 127.24.0.190:42265)
0812 01:19:01.236501 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:01.236503 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:01.236515 (+ 12us) server_negotiation.cc:407] Connection header received
0812 01:19:01.236588 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:01.236589 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:01.236597 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:01.236642 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:01.237224 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:01.237681 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:01.238602 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:01.238784 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:01.238836 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:01.240159 (+ 1323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:01.240166 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:01.240166 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:01.240172 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:01.241586 (+ 1414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:01.241961 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:01.241961 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:01.241962 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:01.242011 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:01.242224 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:01.242225 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:01.242225 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:01.242325 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:01.242396 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:01.242523 (+ 127us) server_negotiation.cc:299] Negotiation successful
0812 01:19:01.242579 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
W20260812 01:19:01.242890 26333 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59896
I20260812 01:19:01.243508 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 26450
I20260812 01:19:01.285707 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 26672
I20260812 01:19:01.298957 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 26918
I20260812 01:19:01.311012 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 26288
2026-08-12T01:19:01Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2563 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 01:19:01 dist-test-slave-t3q3 krb5kdc[27285](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 01:19:01 dist-test-slave-t3q3 krb5kdc[27285](info): set up 2 sockets
Aug 12 01:19:01 dist-test-slave-t3q3 krb5kdc[27285](info): commencing operation
krb5kdc: starting...
W20260812 01:19:02.588299 24578 mini_kdc.cc:121] Time spent starting KDC: real 1.251s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/krb5kdc/test-user.keytab.
Aug 12 01:19:02 dist-test-slave-t3q3 krb5kdc[27285](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497542, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T01:19:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:19:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.0.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.0.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.0.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab.
Entry for principal kudu/127.24.0.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab.
I20260812 01:19:02.710680 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.190:43905
--webserver_interface=127.24.0.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab
--principal=kudu/127.24.0.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:35199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.0.190:43905
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:19:02.799093 27304 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:19:02.799348 27304 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:19:02.799386 27304 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:19:02.800577 27304 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:19:02.800640 27304 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260812 01:19:02.800663 27304 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:19:02.800685 27304 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:19:02.800716 27304 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:19:02.800741 27304 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:19:02.802245 27304 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:35199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.0.190:43905
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab
--principal=kudu/127.24.0.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.190:43905
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.0.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27304
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:19:02.802527 27304 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:19:02.802759 27304 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:19:02.807684 27312 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:02.807732 27313 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:19:02.807850 27304 server_base.cc:1034] running on GCE node
W20260812 01:19:02.807991 27315 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:19:02.808313 27304 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:19:02.808646 27304 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:19:02.809814 27304 hybrid_clock.cc:648] HybridClock initialized: now 1786497542809795 us; error 38 us; skew 500 ppm
Aug 12 01:19:02 dist-test-slave-t3q3 krb5kdc[27285](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497542, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:19:02.812350 27304 init.cc:377] Logged in from keytab as kudu/127.24.0.190@KRBTEST.COM (short username kudu)
I20260812 01:19:02.812844 27304 webserver.cc:460] Webserver started at http://127.24.0.190:36159/ using document root <none> and password file <none>
I20260812 01:19:02.813099 27304 fs_manager.cc:359] Metadata directory not provided
I20260812 01:19:02.813180 27304 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:19:02.813340 27304 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:19:02.814411 27304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/minicluster-data/master-0/data/instance:
uuid: "d3cf79f36f094ec0aee5f37e3cf9fe7c"
format_stamp: "Formatted at 2026-08-12 01:19:02 on dist-test-slave-t3q3"
server_key: "f713640eca089d4180c31fd040ba53f7"
server_key_iv: "e5bf26e28312d71923c69a9abf991061"
server_key_version: "encryptionkey@0"
I20260812 01:19:02.814785 27304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/minicluster-data/master-0/wal/instance:
uuid: "d3cf79f36f094ec0aee5f37e3cf9fe7c"
format_stamp: "Formatted at 2026-08-12 01:19:02 on dist-test-slave-t3q3"
server_key: "f713640eca089d4180c31fd040ba53f7"
server_key_iv: "e5bf26e28312d71923c69a9abf991061"
server_key_version: "encryptionkey@0"
I20260812 01:19:02.816188 27304 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 01:19:02.816959 27327 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:02.817137 27304 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:19:02.817251 27304 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/minicluster-data/master-0/wal
uuid: "d3cf79f36f094ec0aee5f37e3cf9fe7c"
format_stamp: "Formatted at 2026-08-12 01:19:02 on dist-test-slave-t3q3"
server_key: "f713640eca089d4180c31fd040ba53f7"
server_key_iv: "e5bf26e28312d71923c69a9abf991061"
server_key_version: "encryptionkey@0"
I20260812 01:19:02.817327 27304 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:19:02.847676 27304 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:19:02.849813 27304 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:19:02.849989 27304 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:19:02.855113 27304 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.190:43905
I20260812 01:19:02.855170 27426 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.190:43905 every 8 connection(s)
I20260812 01:19:02.855511 27304 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
I20260812 01:19:02.856204 27430 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:19:02.857936 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 27304
I20260812 01:19:02.858036 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/minicluster-data/master-0/wal/instance
I20260812 01:19:02.858208 24578 external_mini_cluster.cc:1428] Setting key dd394e24e022b76baae935fa6a9079dd
I20260812 01:19:02.858915 27430 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3cf79f36f094ec0aee5f37e3cf9fe7c: Bootstrap starting.
I20260812 01:19:02.859510 27430 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d3cf79f36f094ec0aee5f37e3cf9fe7c: Neither blocks nor log segments found. Creating new log.
I20260812 01:19:02.859838 27430 log.cc:824] T 00000000000000000000000000000000 P d3cf79f36f094ec0aee5f37e3cf9fe7c: Log is configured to *not* fsync() on all Append() calls
I20260812 01:19:02.860615 27430 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3cf79f36f094ec0aee5f37e3cf9fe7c: No bootstrap required, opened a new log
I20260812 01:19:02.862551 27430 raft_consensus.cc:348] T 00000000000000000000000000000000 P d3cf79f36f094ec0aee5f37e3cf9fe7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3cf79f36f094ec0aee5f37e3cf9fe7c" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 43905 } }
I20260812 01:19:02.862727 27430 raft_consensus.cc:374] T 00000000000000000000000000000000 P d3cf79f36f094ec0aee5f37e3cf9fe7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:19:02.862800 27430 raft_consensus.cc:729] T 00000000000000000000000000000000 P d3cf79f36f094ec0aee5f37e3cf9fe7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3cf79f36f094ec0aee5f37e3cf9fe7c, State: Initialized, Role: FOLLOWER
I20260812 01:19:02.862970 27430 consensus_queue.cc:260] T 00000000000000000000000000000000 P d3cf79f36f094ec0aee5f37e3cf9fe7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3cf79f36f094ec0aee5f37e3cf9fe7c" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 43905 } }
I20260812 01:19:02.863076 27430 raft_consensus.cc:388] T 00000000000000000000000000000000 P d3cf79f36f094ec0aee5f37e3cf9fe7c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:19:02.863114 27430 raft_consensus.cc:482] T 00000000000000000000000000000000 P d3cf79f36f094ec0aee5f37e3cf9fe7c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:19:02.863169 27430 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d3cf79f36f094ec0aee5f37e3cf9fe7c [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:19:02.863929 27430 raft_consensus.cc:504] T 00000000000000000000000000000000 P d3cf79f36f094ec0aee5f37e3cf9fe7c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3cf79f36f094ec0aee5f37e3cf9fe7c" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 43905 } }
I20260812 01:19:02.864054 27430 leader_election.cc:302] T 00000000000000000000000000000000 P d3cf79f36f094ec0aee5f37e3cf9fe7c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d3cf79f36f094ec0aee5f37e3cf9fe7c; no voters:
I20260812 01:19:02.864264 27430 leader_election.cc:288] T 00000000000000000000000000000000 P d3cf79f36f094ec0aee5f37e3cf9fe7c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 01:19:02.864320 27439 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d3cf79f36f094ec0aee5f37e3cf9fe7c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:19:02.864485 27439 raft_consensus.cc:686] T 00000000000000000000000000000000 P d3cf79f36f094ec0aee5f37e3cf9fe7c [term 1 LEADER]: Becoming Leader. State: Replica: d3cf79f36f094ec0aee5f37e3cf9fe7c, State: Running, Role: LEADER
I20260812 01:19:02.864535 27430 sys_catalog.cc:564] T 00000000000000000000000000000000 P d3cf79f36f094ec0aee5f37e3cf9fe7c [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:19:02.864570 27439 consensus_queue.cc:237] T 00000000000000000000000000000000 P d3cf79f36f094ec0aee5f37e3cf9fe7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3cf79f36f094ec0aee5f37e3cf9fe7c" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 43905 } }
I20260812 01:19:02.865001 27442 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3cf79f36f094ec0aee5f37e3cf9fe7c [sys.catalog]: SysCatalogTable state changed. Reason: New leader d3cf79f36f094ec0aee5f37e3cf9fe7c. Latest consensus state: current_term: 1 leader_uuid: "d3cf79f36f094ec0aee5f37e3cf9fe7c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3cf79f36f094ec0aee5f37e3cf9fe7c" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 43905 } } }
I20260812 01:19:02.865005 27440 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3cf79f36f094ec0aee5f37e3cf9fe7c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d3cf79f36f094ec0aee5f37e3cf9fe7c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3cf79f36f094ec0aee5f37e3cf9fe7c" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 43905 } } }
I20260812 01:19:02.865128 27440 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3cf79f36f094ec0aee5f37e3cf9fe7c [sys.catalog]: This master's current role is: LEADER
I20260812 01:19:02.865123 27442 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3cf79f36f094ec0aee5f37e3cf9fe7c [sys.catalog]: This master's current role is: LEADER
Aug 12 01:19:02 dist-test-slave-t3q3 krb5kdc[27285](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497542, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
I20260812 01:19:02.865825 27446 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:19:02.866420 27446 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:19:02.868299 27446 catalog_manager.cc:1347] Generated new cluster ID: fcddd422f7384dd6a6370c96a2ffefd2
I20260812 01:19:02.868353 27446 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 01:19:02.869666 27435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:02.858742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34752 (local address 127.24.0.190:43905)
0812 01:19:02.858933 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:02.858941 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:02.858985 (+ 44us) server_negotiation.cc:407] Connection header received
0812 01:19:02.859449 (+ 464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:02.859455 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:02.859501 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:02.859658 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:02.860589 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:02.861572 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:02.862330 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:02.862778 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:02.862883 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:02.865792 (+ 2909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:02.865796 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:02.865797 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:02.865809 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:02.867940 (+ 2131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:02.868344 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:02.868347 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:02.868348 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:02.868505 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:02.868769 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:02.868770 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:02.868770 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:02.869054 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:02.869178 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:02.869340 (+ 162us) server_negotiation.cc:299] Negotiation successful
0812 01:19:02.869468 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":83,"threads_started":1}
I20260812 01:19:02.881702 27446 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:19:02.882225 27446 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:19:02.892284 27446 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d3cf79f36f094ec0aee5f37e3cf9fe7c: Generated new TSK 0
I20260812 01:19:02.892474 27446 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.0.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.0.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.0.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.129.keytab.
Entry for principal kudu/127.24.0.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.129.keytab.
I20260812 01:19:02.927448 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.0.129:0
--local_ip_for_outbound_sockets=127.24.0.129
--webserver_interface=127.24.0.129
--webserver_port=0
--tserver_master_addrs=127.24.0.190:43905
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.129.keytab
--principal=kudu/127.24.0.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:35199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:19:03.020196 27469 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:19:03.020434 27469 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:19:03.020459 27469 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:19:03.021531 27469 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:19:03.021569 27469 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:19:03.021588 27469 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260812 01:19:03.021600 27469 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 01:19:03.021618 27469 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.0.129
I20260812 01:19:03.023001 27469 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:35199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.129.keytab
--principal=kudu/127.24.0.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.0.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.0.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.0.190:43905
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27469
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.0.129
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:19:03.023245 27469 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:19:03.023435 27469 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:19:03.028445 27481 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:03.028436 27488 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:03.028437 27484 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:19:03.028596 27469 server_base.cc:1034] running on GCE node
I20260812 01:19:03.028805 27469 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:19:03.029054 27469 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:19:03.030232 27469 hybrid_clock.cc:648] HybridClock initialized: now 1786497543030226 us; error 37 us; skew 500 ppm
Aug 12 01:19:03 dist-test-slave-t3q3 krb5kdc[27285](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497543, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:19:03.036773 27469 init.cc:377] Logged in from keytab as kudu/127.24.0.129@KRBTEST.COM (short username kudu)
I20260812 01:19:03.037235 27469 webserver.cc:460] Webserver started at http://127.24.0.129:40233/ using document root <none> and password file <none>
I20260812 01:19:03.037452 27469 fs_manager.cc:359] Metadata directory not provided
I20260812 01:19:03.037492 27469 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:19:03.037586 27469 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:19:03.038597 27469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/minicluster-data/ts-0/data/instance:
uuid: "ff2989b8d8bd4cc9ab32b320fbd3d52d"
format_stamp: "Formatted at 2026-08-12 01:19:03 on dist-test-slave-t3q3"
server_key: "3c48dbc2e5e95542ec1e482159094d9c"
server_key_iv: "b589d758a70a185b6413bbee1c18b76e"
server_key_version: "encryptionkey@0"
I20260812 01:19:03.038911 27469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/minicluster-data/ts-0/wal/instance:
uuid: "ff2989b8d8bd4cc9ab32b320fbd3d52d"
format_stamp: "Formatted at 2026-08-12 01:19:03 on dist-test-slave-t3q3"
server_key: "3c48dbc2e5e95542ec1e482159094d9c"
server_key_iv: "b589d758a70a185b6413bbee1c18b76e"
server_key_version: "encryptionkey@0"
I20260812 01:19:03.040213 27469 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 01:19:03.040975 27497 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:03.041155 27469 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 01:19:03.041253 27469 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/minicluster-data/ts-0/wal
uuid: "ff2989b8d8bd4cc9ab32b320fbd3d52d"
format_stamp: "Formatted at 2026-08-12 01:19:03 on dist-test-slave-t3q3"
server_key: "3c48dbc2e5e95542ec1e482159094d9c"
server_key_iv: "b589d758a70a185b6413bbee1c18b76e"
server_key_version: "encryptionkey@0"
I20260812 01:19:03.041309 27469 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:19:03.064390 27469 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:19:03.066542 27469 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:19:03.066665 27469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:19:03.067225 27469 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:19:03.067270 27469 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:03.067297 27469 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:19:03.067345 27469 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:03.074150 27469 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.129:45255
I20260812 01:19:03.074237 27684 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.129:45255 every 8 connection(s)
Aug 12 01:19:03 dist-test-slave-t3q3 krb5kdc[27285](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497543, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.129@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
I20260812 01:19:03.074604 27469 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/minicluster-data/ts-0/data/info.pb
I20260812 01:19:03.077677 27435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:03.067791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.129:60691 (local address 127.24.0.190:43905)
0812 01:19:03.067874 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:03.067878 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:03.068091 (+ 213us) server_negotiation.cc:407] Connection header received
0812 01:19:03.068621 (+ 530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:03.068623 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:03.068637 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:03.068737 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:03.069592 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:03.070454 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:03.072111 (+ 1657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:03.072347 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:03.072435 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:03.075061 (+ 2626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:03.075063 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:03.075063 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:03.075071 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:03.076459 (+ 1388us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:03.076860 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:03.076861 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:03.076861 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:03.076915 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:03.077132 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:03.077132 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:03.077133 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:03.077248 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:03.077300 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:03.077535 (+ 235us) server_negotiation.cc:299] Negotiation successful
0812 01:19:03.077609 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 01:19:03.077836 27518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:03.067748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:43905 (local address 127.24.0.129:60691)
0812 01:19:03.067937 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:03.067978 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:03.068512 (+ 534us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:03.068793 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:03.068796 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:03.069200 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:03.069498 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:03.069505 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:03.070550 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:03.070551 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:03.072005 (+ 1454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:03.072008 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:03.072450 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:03.072451 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:03.072620 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:03.073294 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:03.073304 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:03.074948 (+ 1644us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:19:03.076537 (+ 1589us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:03.076543 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:03.076545 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:03.076790 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:03.076974 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:03.076975 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:03.076975 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:03.077077 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:03.077305 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:19:03.077310 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:19:03.077427 (+ 117us) client_negotiation.cc:769] Sending connection context
0812 01:19:03.077510 (+ 83us) client_negotiation.cc:240] Negotiation successful
0812 01:19:03.077632 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":113,"threads_started":1}
I20260812 01:19:03.078380 27691 heartbeater.cc:347] Connected to a master server at 127.24.0.190:43905
I20260812 01:19:03.078485 27691 heartbeater.cc:464] Registering TS with master...
I20260812 01:19:03.078682 27691 heartbeater.cc:511] Master 127.24.0.190:43905 requested a full tablet report, sending...
I20260812 01:19:03.079273 27353 ts_manager.cc:194] Registered new tserver with Master: ff2989b8d8bd4cc9ab32b320fbd3d52d (127.24.0.129:45255)
I20260812 01:19:03.079938 27353 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.0.129@KRBTEST.COM'} at 127.24.0.129:60691
I20260812 01:19:03.084013 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 27469
I20260812 01:19:03.084152 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/minicluster-data/ts-0/wal/instance
I20260812 01:19:03.084333 24578 external_mini_cluster.cc:1428] Setting key 1662f1e8cfc37f68c634620b732367b6
I20260812 01:19:03.085193 27699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:03.079414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:43905 (local address 127.24.0.129:52211)
0812 01:19:03.079532 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:03.079543 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:03.079593 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:03.080290 (+ 697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:03.080291 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:03.080484 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:03.080534 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:03.080535 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:03.081179 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:03.081180 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:03.081673 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:03.081674 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:03.082059 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:03.082060 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:03.082140 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:03.082498 (+ 358us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:03.082502 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:03.082779 (+ 277us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:19:03.084099 (+ 1320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:03.084100 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:03.084101 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:03.084313 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:03.084534 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:03.084534 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:03.084535 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:03.084595 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:03.084859 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:19:03.084860 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:19:03.084906 (+ 46us) client_negotiation.cc:769] Sending connection context
0812 01:19:03.084985 (+ 79us) client_negotiation.cc:240] Negotiation successful
0812 01:19:03.085093 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":62,"threads_started":1}
I20260812 01:19:03.085184 27435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:03.080117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.129:52211 (local address 127.24.0.190:43905)
0812 01:19:03.080162 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:03.080164 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:03.080171 (+ 7us) server_negotiation.cc:407] Connection header received
0812 01:19:03.080201 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:03.080201 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:03.080210 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:03.080269 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:03.080619 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:03.081086 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:03.081764 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:03.081973 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:03.082057 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:03.082875 (+ 818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:03.082877 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:03.082877 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:03.082883 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:03.083976 (+ 1093us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:03.084402 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:03.084403 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:03.084404 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:03.084466 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:03.084668 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:03.084669 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:03.084670 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:03.084783 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:03.084857 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:03.085024 (+ 167us) server_negotiation.cc:299] Negotiation successful
0812 01:19:03.085095 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
WARNING: no policy specified for kudu/127.24.0.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.0.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.0.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.130.keytab.
Entry for principal kudu/127.24.0.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.130.keytab.
I20260812 01:19:03.111500 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.0.130:0
--local_ip_for_outbound_sockets=127.24.0.130
--webserver_interface=127.24.0.130
--webserver_port=0
--tserver_master_addrs=127.24.0.190:43905
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.130.keytab
--principal=kudu/127.24.0.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:35199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:19:03.196935 27702 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:19:03.197146 27702 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:19:03.197178 27702 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:19:03.198323 27702 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:19:03.198369 27702 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:19:03.198388 27702 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260812 01:19:03.198400 27702 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 01:19:03.198417 27702 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.0.130
I20260812 01:19:03.199913 27702 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:35199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.130.keytab
--principal=kudu/127.24.0.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.0.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.0.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.0.190:43905
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27702
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.0.130
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:19:03.200215 27702 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:19:03.200474 27702 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:19:03.205389 27717 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:03.205470 27714 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:03.205392 27713 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:19:03.205897 27702 server_base.cc:1034] running on GCE node
I20260812 01:19:03.206127 27702 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:19:03.206363 27702 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:19:03.207506 27702 hybrid_clock.cc:648] HybridClock initialized: now 1786497543207486 us; error 41 us; skew 500 ppm
Aug 12 01:19:03 dist-test-slave-t3q3 krb5kdc[27285](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497543, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:19:03.210079 27702 init.cc:377] Logged in from keytab as kudu/127.24.0.130@KRBTEST.COM (short username kudu)
I20260812 01:19:03.210511 27702 webserver.cc:460] Webserver started at http://127.24.0.130:33721/ using document root <none> and password file <none>
I20260812 01:19:03.210758 27702 fs_manager.cc:359] Metadata directory not provided
I20260812 01:19:03.210826 27702 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:19:03.210975 27702 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:19:03.212016 27702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/minicluster-data/ts-1/data/instance:
uuid: "5969fd3075dd4de192f2fc1e163a42b0"
format_stamp: "Formatted at 2026-08-12 01:19:03 on dist-test-slave-t3q3"
server_key: "83113051fb89c06d77195a8a7f355979"
server_key_iv: "08b3d5abfb03ae0eca1f4d5572b66f7f"
server_key_version: "encryptionkey@0"
I20260812 01:19:03.212446 27702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/minicluster-data/ts-1/wal/instance:
uuid: "5969fd3075dd4de192f2fc1e163a42b0"
format_stamp: "Formatted at 2026-08-12 01:19:03 on dist-test-slave-t3q3"
server_key: "83113051fb89c06d77195a8a7f355979"
server_key_iv: "08b3d5abfb03ae0eca1f4d5572b66f7f"
server_key_version: "encryptionkey@0"
I20260812 01:19:03.213896 27702 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:19:03.214695 27728 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:03.214903 27702 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:19:03.215011 27702 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/minicluster-data/ts-1/wal
uuid: "5969fd3075dd4de192f2fc1e163a42b0"
format_stamp: "Formatted at 2026-08-12 01:19:03 on dist-test-slave-t3q3"
server_key: "83113051fb89c06d77195a8a7f355979"
server_key_iv: "08b3d5abfb03ae0eca1f4d5572b66f7f"
server_key_version: "encryptionkey@0"
I20260812 01:19:03.215086 27702 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:19:03.242301 27702 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:19:03.244874 27702 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:19:03.245052 27702 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:19:03.245615 27702 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:19:03.245689 27702 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:03.245752 27702 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:19:03.245784 27702 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:03.252719 27702 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.130:41963
Aug 12 01:19:03 dist-test-slave-t3q3 krb5kdc[27285](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497543, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.130@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
I20260812 01:19:03.252805 27937 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.130:41963 every 8 connection(s)
I20260812 01:19:03.253126 27702 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/minicluster-data/ts-1/data/info.pb
I20260812 01:19:03.255975 27435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:03.246226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.130:51023 (local address 127.24.0.190:43905)
0812 01:19:03.246275 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:03.246277 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:03.246500 (+ 223us) server_negotiation.cc:407] Connection header received
0812 01:19:03.247044 (+ 544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:03.247046 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:03.247061 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:03.247111 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:03.248049 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:03.248712 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:03.250238 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:03.250560 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:03.250633 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:03.253360 (+ 2727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:03.253362 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:03.253363 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:03.253372 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:03.254819 (+ 1447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:03.255189 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:03.255189 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:03.255190 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:03.255242 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:03.255455 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:03.255455 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:03.255456 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:03.255560 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:03.255613 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:03.255844 (+ 231us) server_negotiation.cc:299] Negotiation successful
0812 01:19:03.255909 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 01:19:03.256194 27744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:03.246207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:43905 (local address 127.24.0.130:51023)
0812 01:19:03.246373 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:03.246408 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:03.246934 (+ 526us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:03.247203 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:03.247209 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:03.247642 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:03.247958 (+ 316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:03.247965 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:03.248808 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:03.248809 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:03.250155 (+ 1346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:03.250159 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:03.250659 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:03.250660 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:03.250829 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:03.251424 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:03.251434 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:03.253233 (+ 1799us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:19:03.254896 (+ 1663us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:03.254902 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:03.254904 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:03.255116 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:03.255301 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:03.255302 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:03.255302 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:03.255399 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:03.255616 (+ 217us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:19:03.255621 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:19:03.255731 (+ 110us) client_negotiation.cc:769] Sending connection context
0812 01:19:03.255815 (+ 84us) client_negotiation.cc:240] Negotiation successful
0812 01:19:03.255934 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":96,"threads_started":1}
I20260812 01:19:03.256733 27938 heartbeater.cc:347] Connected to a master server at 127.24.0.190:43905
I20260812 01:19:03.256841 27938 heartbeater.cc:464] Registering TS with master...
I20260812 01:19:03.257050 27938 heartbeater.cc:511] Master 127.24.0.190:43905 requested a full tablet report, sending...
I20260812 01:19:03.257448 27353 ts_manager.cc:194] Registered new tserver with Master: 5969fd3075dd4de192f2fc1e163a42b0 (127.24.0.130:41963)
I20260812 01:19:03.257722 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 27702
I20260812 01:19:03.257833 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/minicluster-data/ts-1/wal/instance
I20260812 01:19:03.257889 27353 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.0.130@KRBTEST.COM'} at 127.24.0.130:51023
I20260812 01:19:03.258039 24578 external_mini_cluster.cc:1428] Setting key a93b1a7bd1a3ea475d3370a0551f7353
I20260812 01:19:03.264312 27953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:03.258045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:43905 (local address 127.24.0.130:57873)
0812 01:19:03.258245 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:03.258256 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:03.258315 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:03.258436 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:03.258437 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:03.258631 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:03.258688 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:03.258690 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:03.259526 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:03.259527 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:03.260008 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:03.260009 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:03.260516 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:03.260517 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:03.260599 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:03.260999 (+ 400us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:03.261005 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:03.261418 (+ 413us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:19:03.263174 (+ 1756us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:03.263175 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:03.263175 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:03.263344 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:03.263572 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:03.263572 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:03.263573 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:03.263624 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:03.263990 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:19:03.263990 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:19:03.264028 (+ 38us) client_negotiation.cc:769] Sending connection context
0812 01:19:03.264148 (+ 120us) client_negotiation.cc:240] Negotiation successful
0812 01:19:03.264246 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":133,"threads_started":1}
I20260812 01:19:03.264362 27435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:03.257919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.130:57873 (local address 127.24.0.190:43905)
0812 01:19:03.257961 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:03.257963 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:03.258304 (+ 341us) server_negotiation.cc:407] Connection header received
0812 01:19:03.258361 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:03.258362 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:03.258372 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:03.258422 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:03.258759 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:03.259429 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:03.260124 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:03.260429 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:03.260507 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:03.261507 (+ 1000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:03.261508 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:03.261508 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:03.261514 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:03.263094 (+ 1580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:03.263424 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:03.263425 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:03.263426 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:03.263499 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:03.263694 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:03.263695 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:03.263695 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:03.263914 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:03.263994 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:03.264177 (+ 183us) server_negotiation.cc:299] Negotiation successful
0812 01:19:03.264269 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
WARNING: no policy specified for kudu/127.24.0.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.0.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.0.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.131.keytab.
Entry for principal kudu/127.24.0.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.131.keytab.
I20260812 01:19:03.286489 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.0.131:0
--local_ip_for_outbound_sockets=127.24.0.131
--webserver_interface=127.24.0.131
--webserver_port=0
--tserver_master_addrs=127.24.0.190:43905
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.131.keytab
--principal=kudu/127.24.0.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:35199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:19:03.371933 27957 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:19:03.372236 27957 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:19:03.372267 27957 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:19:03.373447 27957 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:19:03.373489 27957 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:19:03.373539 27957 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260812 01:19:03.373567 27957 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 01:19:03.373595 27957 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.0.131
I20260812 01:19:03.375051 27957 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:35199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.131.keytab
--principal=kudu/127.24.0.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.0.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.0.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.0.190:43905
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27957
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.0.131
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:19:03.375310 27957 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:19:03.375561 27957 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 01:19:03.380846 27957 server_base.cc:1034] running on GCE node
W20260812 01:19:03.380852 27965 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:03.380970 27967 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:03.380862 27964 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:19:03.381296 27957 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:19:03.381551 27957 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:19:03.382735 27957 hybrid_clock.cc:648] HybridClock initialized: now 1786497543382724 us; error 40 us; skew 500 ppm
Aug 12 01:19:03 dist-test-slave-t3q3 krb5kdc[27285](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497543, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:19:03.385566 27957 init.cc:377] Logged in from keytab as kudu/127.24.0.131@KRBTEST.COM (short username kudu)
I20260812 01:19:03.385987 27957 webserver.cc:460] Webserver started at http://127.24.0.131:37779/ using document root <none> and password file <none>
I20260812 01:19:03.386233 27957 fs_manager.cc:359] Metadata directory not provided
I20260812 01:19:03.386303 27957 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:19:03.386456 27957 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:19:03.387497 27957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/minicluster-data/ts-2/data/instance:
uuid: "1da32ab00b8748aeb0b9879201a73b6d"
format_stamp: "Formatted at 2026-08-12 01:19:03 on dist-test-slave-t3q3"
server_key: "80fb7cd85943b56903371a7b99039ec5"
server_key_iv: "8fceda8039383645a271873b761078ea"
server_key_version: "encryptionkey@0"
I20260812 01:19:03.387868 27957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/minicluster-data/ts-2/wal/instance:
uuid: "1da32ab00b8748aeb0b9879201a73b6d"
format_stamp: "Formatted at 2026-08-12 01:19:03 on dist-test-slave-t3q3"
server_key: "80fb7cd85943b56903371a7b99039ec5"
server_key_iv: "8fceda8039383645a271873b761078ea"
server_key_version: "encryptionkey@0"
I20260812 01:19:03.389318 27957 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:19:03.390101 27977 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:03.390307 27957 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:19:03.390409 27957 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/minicluster-data/ts-2/wal
uuid: "1da32ab00b8748aeb0b9879201a73b6d"
format_stamp: "Formatted at 2026-08-12 01:19:03 on dist-test-slave-t3q3"
server_key: "80fb7cd85943b56903371a7b99039ec5"
server_key_iv: "8fceda8039383645a271873b761078ea"
server_key_version: "encryptionkey@0"
I20260812 01:19:03.390493 27957 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:19:03.403409 27957 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:19:03.405819 27957 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:19:03.405982 27957 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:19:03.406509 27957 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:19:03.406576 27957 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:03.406623 27957 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:19:03.406647 27957 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 01:19:03 dist-test-slave-t3q3 krb5kdc[27285](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497543, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.131@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
I20260812 01:19:03.414100 27957 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.131:34529
I20260812 01:19:03.414215 28177 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.131:34529 every 8 connection(s)
I20260812 01:19:03.414520 27957 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/minicluster-data/ts-2/data/info.pb
I20260812 01:19:03.417258 27435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:03.407016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.131:42835 (local address 127.24.0.190:43905)
0812 01:19:03.407093 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:03.407097 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:03.407331 (+ 234us) server_negotiation.cc:407] Connection header received
0812 01:19:03.407917 (+ 586us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:03.407919 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:03.407935 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:03.408023 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:03.408999 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:03.409844 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:03.411490 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:03.411849 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:03.411964 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:03.414685 (+ 2721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:03.414687 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:03.414688 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:03.414698 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:03.416024 (+ 1326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:03.416465 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:03.416466 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:03.416467 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:03.416522 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:03.416739 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:03.416740 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:03.416740 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:03.416847 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:03.416900 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:03.417128 (+ 228us) server_negotiation.cc:299] Negotiation successful
0812 01:19:03.417192 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 01:19:03.417440 27991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:03.406999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:43905 (local address 127.24.0.131:42835)
0812 01:19:03.407191 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:03.407231 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:03.407767 (+ 536us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:03.408135 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:03.408139 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:03.408565 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:03.408900 (+ 335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:03.408907 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:03.409975 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:03.409976 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:03.411382 (+ 1406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:03.411386 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:03.412011 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:03.412013 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:03.412186 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:03.412796 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:03.412805 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:03.414589 (+ 1784us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:19:03.416163 (+ 1574us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:03.416176 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:03.416179 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:03.416399 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:03.416582 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:03.416583 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:03.416583 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:03.416678 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:03.416903 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:19:03.416907 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:19:03.417017 (+ 110us) client_negotiation.cc:769] Sending connection context
0812 01:19:03.417098 (+ 81us) client_negotiation.cc:240] Negotiation successful
0812 01:19:03.417217 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":113,"threads_started":1}
I20260812 01:19:03.417968 28180 heartbeater.cc:347] Connected to a master server at 127.24.0.190:43905
I20260812 01:19:03.418056 28180 heartbeater.cc:464] Registering TS with master...
I20260812 01:19:03.418267 28180 heartbeater.cc:511] Master 127.24.0.190:43905 requested a full tablet report, sending...
I20260812 01:19:03.418735 27353 ts_manager.cc:194] Registered new tserver with Master: 1da32ab00b8748aeb0b9879201a73b6d (127.24.0.131:34529)
I20260812 01:19:03.419209 27353 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.0.131@KRBTEST.COM'} at 127.24.0.131:42835
I20260812 01:19:03.422910 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 27957
I20260812 01:19:03.422986 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/minicluster-data/ts-2/wal/instance
I20260812 01:19:03.423192 24578 external_mini_cluster.cc:1428] Setting key aad156f273699f43291d3051b329b4ef
I20260812 01:19:03.423992 24578 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 01:19:03.424676 27435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:03.419153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.131:54087 (local address 127.24.0.190:43905)
0812 01:19:03.419200 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:03.419201 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:03.419323 (+ 122us) server_negotiation.cc:407] Connection header received
0812 01:19:03.419396 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:03.419397 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:03.419408 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:03.419457 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:03.419834 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:03.420374 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:03.421010 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:03.421182 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:03.421233 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:03.422004 (+ 771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:03.422005 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:03.422006 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:03.422012 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:03.423498 (+ 1486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:03.423864 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:03.423866 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:03.423866 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:03.423927 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:03.424159 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:03.424160 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:03.424160 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:03.424276 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:03.424349 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:03.424512 (+ 163us) server_negotiation.cc:299] Negotiation successful
0812 01:19:03.424586 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 01:19:03.424692 28188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:03.419083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:43905 (local address 127.24.0.131:54087)
0812 01:19:03.419247 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:03.419264 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:03.419337 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:03.419459 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:03.419460 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:03.419674 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:03.419769 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:03.419771 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:03.420460 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:03.420460 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:03.420936 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:03.420938 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:03.421250 (+ 312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:03.421250 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:03.421309 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:03.421652 (+ 343us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:03.421656 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:03.421928 (+ 272us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:19:03.423586 (+ 1658us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:03.423587 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:03.423587 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:03.423774 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:03.423999 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:03.424000 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:03.424000 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:03.424103 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:03.424354 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:19:03.424355 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:19:03.424400 (+ 45us) client_negotiation.cc:769] Sending connection context
0812 01:19:03.424486 (+ 86us) client_negotiation.cc:240] Negotiation successful
0812 01:19:03.424588 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":83,"threads_started":1}
Aug 12 01:19:03 dist-test-slave-t3q3 krb5kdc[27285](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497543, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 01:19:03 dist-test-slave-t3q3 krb5kdc[27285](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497543, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
I20260812 01:19:03.448467 27435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:03.441439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34760 (local address 127.24.0.190:43905)
0812 01:19:03.441505 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:03.441508 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:03.441517 (+ 9us) server_negotiation.cc:407] Connection header received
0812 01:19:03.441553 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:03.441554 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:03.441563 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:03.441615 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:03.442165 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:03.442661 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:03.443404 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:03.443591 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:03.443645 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:03.445985 (+ 2340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:03.445986 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:03.445986 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:03.445993 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:03.447395 (+ 1402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:03.447736 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:03.447737 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:03.447737 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:03.447791 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:03.447965 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:03.447966 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:03.447967 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:03.448123 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:03.448219 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:03.448312 (+ 93us) server_negotiation.cc:299] Negotiation successful
0812 01:19:03.448375 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 01:19:03.450042 27353 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34760:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 01:19:03.451653 27353 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 01:19:03.458626 28205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:03.454006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.131:34529 (local address 127.0.0.1:39502)
0812 01:19:03.454362 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:03.454405 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:03.454504 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:03.455016 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:03.455017 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:03.455019 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:19:03.455074 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:03.455076 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:03.455966 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:03.455967 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:03.457541 (+ 1574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:03.457544 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:03.458280 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:03.458281 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:03.458372 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:03.458386 (+ 14us) client_negotiation.cc:769] Sending connection context
0812 01:19:03.458484 (+ 98us) client_negotiation.cc:240] Negotiation successful
0812 01:19:03.458539 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":322,"mutex_wait_us":100,"thread_start_us":59,"threads_started":1}
I20260812 01:19:03.458634 28207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:03.454134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39502 (local address 127.24.0.131:34529)
0812 01:19:03.454373 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:03.454376 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:03.454520 (+ 144us) server_negotiation.cc:407] Connection header received
0812 01:19:03.454843 (+ 323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:03.454846 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:03.454863 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:03.454942 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:19:03.455147 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:03.455866 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:03.457666 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:03.458183 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:03.458280 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:03.458346 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:03.458503 (+ 157us) server_negotiation.cc:299] Negotiation successful
0812 01:19:03.458544 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":116,"threads_started":1}
I20260812 01:19:03.458851 28210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:03.454231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.130:41963 (local address 127.0.0.1:49626)
0812 01:19:03.454625 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:03.454633 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:03.454718 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:03.455106 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:03.455106 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:03.455108 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:19:03.455150 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:03.455152 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:03.456427 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:03.456428 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:03.457554 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:03.457557 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:03.458579 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:03.458580 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:03.458642 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:03.458646 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 01:19:03.458700 (+ 54us) client_negotiation.cc:240] Negotiation successful
0812 01:19:03.458746 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":75,"threads_started":1}
I20260812 01:19:03.459143 28211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:03.454384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49626 (local address 127.24.0.130:41963)
0812 01:19:03.454579 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:03.454584 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:03.454895 (+ 311us) server_negotiation.cc:407] Connection header received
0812 01:19:03.454943 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:03.454946 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:03.454961 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:03.455036 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:19:03.455228 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:03.456339 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:03.457666 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:03.458404 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:03.458480 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:03.458541 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:03.459001 (+ 460us) server_negotiation.cc:299] Negotiation successful
0812 01:19:03.459046 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":86,"threads_started":1}
I20260812 01:19:03.459518 28206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:03.454104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35958 (local address 127.24.0.129:45255)
0812 01:19:03.454293 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:03.454298 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:03.454489 (+ 191us) server_negotiation.cc:407] Connection header received
0812 01:19:03.454600 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:03.454603 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:03.454618 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:03.454719 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:19:03.454873 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:03.456135 (+ 1262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:03.457633 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:03.458461 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:03.458520 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:03.458571 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:03.459027 (+ 456us) server_negotiation.cc:299] Negotiation successful
0812 01:19:03.459438 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":89,"threads_started":1}
I20260812 01:19:03.459451 28059 tablet_service.cc:1467] Processing CreateTablet for tablet 0582dc470a2f4985bdbc2cbd8b7e0e4d (DEFAULT_TABLE table=test-table [id=b7a7087ccd054a429dc7a988aa2923d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:19:03.459760 28203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:03.454006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.129:45255 (local address 127.0.0.1:35958)
0812 01:19:03.454364 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:03.454407 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:03.454505 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:03.454730 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:03.454733 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:03.454735 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:19:03.454806 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:03.454808 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:03.456224 (+ 1416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:03.456225 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:03.457541 (+ 1316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:03.457544 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:03.458600 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:03.458601 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:03.458655 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:03.458658 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 01:19:03.459624 (+ 966us) client_negotiation.cc:240] Negotiation successful
0812 01:19:03.459684 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":338,"mutex_wait_us":41,"thread_start_us":85,"threads_started":1}
I20260812 01:19:03.459898 28059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0582dc470a2f4985bdbc2cbd8b7e0e4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:19:03.460511 27580 tablet_service.cc:1467] Processing CreateTablet for tablet 0582dc470a2f4985bdbc2cbd8b7e0e4d (DEFAULT_TABLE table=test-table [id=b7a7087ccd054a429dc7a988aa2923d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:19:03.460644 27824 tablet_service.cc:1467] Processing CreateTablet for tablet 0582dc470a2f4985bdbc2cbd8b7e0e4d (DEFAULT_TABLE table=test-table [id=b7a7087ccd054a429dc7a988aa2923d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:19:03.460870 27580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0582dc470a2f4985bdbc2cbd8b7e0e4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:19:03.461042 27824 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0582dc470a2f4985bdbc2cbd8b7e0e4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:19:03.463461 28212 tablet_bootstrap.cc:492] T 0582dc470a2f4985bdbc2cbd8b7e0e4d P 1da32ab00b8748aeb0b9879201a73b6d: Bootstrap starting.
I20260812 01:19:03.463881 28213 tablet_bootstrap.cc:492] T 0582dc470a2f4985bdbc2cbd8b7e0e4d P 5969fd3075dd4de192f2fc1e163a42b0: Bootstrap starting.
I20260812 01:19:03.464594 28212 tablet_bootstrap.cc:654] T 0582dc470a2f4985bdbc2cbd8b7e0e4d P 1da32ab00b8748aeb0b9879201a73b6d: Neither blocks nor log segments found. Creating new log.
I20260812 01:19:03.464645 28213 tablet_bootstrap.cc:654] T 0582dc470a2f4985bdbc2cbd8b7e0e4d P 5969fd3075dd4de192f2fc1e163a42b0: Neither blocks nor log segments found. Creating new log.
I20260812 01:19:03.464808 28214 tablet_bootstrap.cc:492] T 0582dc470a2f4985bdbc2cbd8b7e0e4d P ff2989b8d8bd4cc9ab32b320fbd3d52d: Bootstrap starting.
I20260812 01:19:03.464983 28213 log.cc:824] T 0582dc470a2f4985bdbc2cbd8b7e0e4d P 5969fd3075dd4de192f2fc1e163a42b0: Log is configured to *not* fsync() on all Append() calls
I20260812 01:19:03.465312 28212 log.cc:824] T 0582dc470a2f4985bdbc2cbd8b7e0e4d P 1da32ab00b8748aeb0b9879201a73b6d: Log is configured to *not* fsync() on all Append() calls
I20260812 01:19:03.465654 28214 tablet_bootstrap.cc:654] T 0582dc470a2f4985bdbc2cbd8b7e0e4d P ff2989b8d8bd4cc9ab32b320fbd3d52d: Neither blocks nor log segments found. Creating new log.
I20260812 01:19:03.465863 28213 tablet_bootstrap.cc:492] T 0582dc470a2f4985bdbc2cbd8b7e0e4d P 5969fd3075dd4de192f2fc1e163a42b0: No bootstrap required, opened a new log
I20260812 01:19:03.465920 28213 ts_tablet_manager.cc:1397] T 0582dc470a2f4985bdbc2cbd8b7e0e4d P 5969fd3075dd4de192f2fc1e163a42b0: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260812 01:19:03.465996 28214 log.cc:824] T 0582dc470a2f4985bdbc2cbd8b7e0e4d P ff2989b8d8bd4cc9ab32b320fbd3d52d: Log is configured to *not* fsync() on all Append() calls
I20260812 01:19:03.466733 28214 tablet_bootstrap.cc:492] T 0582dc470a2f4985bdbc2cbd8b7e0e4d P ff2989b8d8bd4cc9ab32b320fbd3d52d: No bootstrap required, opened a new log
I20260812 01:19:03.466807 28214 ts_tablet_manager.cc:1397] T 0582dc470a2f4985bdbc2cbd8b7e0e4d P ff2989b8d8bd4cc9ab32b320fbd3d52d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 01:19:03.467454 28213 raft_consensus.cc:348] T 0582dc470a2f4985bdbc2cbd8b7e0e4d P 5969fd3075dd4de192f2fc1e163a42b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff2989b8d8bd4cc9ab32b320fbd3d52d" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 45255 } } peers { permanent_uuid: "1da32ab00b8748aeb0b9879201a73b6d" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34529 } } peers { permanent_uuid: "5969fd3075dd4de192f2fc1e163a42b0" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 41963 } }
I20260812 01:19:03.467583 28213 raft_consensus.cc:374] T 0582dc470a2f4985bdbc2cbd8b7e0e4d P 5969fd3075dd4de192f2fc1e163a42b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:19:03.467604 28213 raft_consensus.cc:729] T 0582dc470a2f4985bdbc2cbd8b7e0e4d P 5969fd3075dd4de192f2fc1e163a42b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5969fd3075dd4de192f2fc1e163a42b0, State: Initialized, Role: FOLLOWER
I20260812 01:19:03.467724 28213 consensus_queue.cc:260] T 0582dc470a2f4985bdbc2cbd8b7e0e4d P 5969fd3075dd4de192f2fc1e163a42b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff2989b8d8bd4cc9ab32b320fbd3d52d" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 45255 } } peers { permanent_uuid: "1da32ab00b8748aeb0b9879201a73b6d" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34529 } } peers { permanent_uuid: "5969fd3075dd4de192f2fc1e163a42b0" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 41963 } }
I20260812 01:19:03.468007 28213 ts_tablet_manager.cc:1428] T 0582dc470a2f4985bdbc2cbd8b7e0e4d P 5969fd3075dd4de192f2fc1e163a42b0: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260812 01:19:03.468101 27938 heartbeater.cc:503] Master 127.24.0.190:43905 was elected leader, sending a full tablet report...
I20260812 01:19:03.468578 28214 raft_consensus.cc:348] T 0582dc470a2f4985bdbc2cbd8b7e0e4d P ff2989b8d8bd4cc9ab32b320fbd3d52d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff2989b8d8bd4cc9ab32b320fbd3d52d" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 45255 } } peers { permanent_uuid: "1da32ab00b8748aeb0b9879201a73b6d" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34529 } } peers { permanent_uuid: "5969fd3075dd4de192f2fc1e163a42b0" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 41963 } }
I20260812 01:19:03.468664 28212 tablet_bootstrap.cc:492] T 0582dc470a2f4985bdbc2cbd8b7e0e4d P 1da32ab00b8748aeb0b9879201a73b6d: No bootstrap required, opened a new log
I20260812 01:19:03.468729 28214 raft_consensus.cc:374] T 0582dc470a2f4985bdbc2cbd8b7e0e4d P ff2989b8d8bd4cc9ab32b320fbd3d52d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:19:03.468750 28212 ts_tablet_manager.cc:1397] T 0582dc470a2f4985bdbc2cbd8b7e0e4d P 1da32ab00b8748aeb0b9879201a73b6d: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260812 01:19:03.468772 28214 raft_consensus.cc:729] T 0582dc470a2f4985bdbc2cbd8b7e0e4d P ff2989b8d8bd4cc9ab32b320fbd3d52d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff2989b8d8bd4cc9ab32b320fbd3d52d, State: Initialized, Role: FOLLOWER
I20260812 01:19:03.468909 28214 consensus_queue.cc:260] T 0582dc470a2f4985bdbc2cbd8b7e0e4d P ff2989b8d8bd4cc9ab32b320fbd3d52d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff2989b8d8bd4cc9ab32b320fbd3d52d" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 45255 } } peers { permanent_uuid: "1da32ab00b8748aeb0b9879201a73b6d" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34529 } } peers { permanent_uuid: "5969fd3075dd4de192f2fc1e163a42b0" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 41963 } }
I20260812 01:19:03.469193 28214 ts_tablet_manager.cc:1428] T 0582dc470a2f4985bdbc2cbd8b7e0e4d P ff2989b8d8bd4cc9ab32b320fbd3d52d: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 01:19:03.469285 27691 heartbeater.cc:503] Master 127.24.0.190:43905 was elected leader, sending a full tablet report...
I20260812 01:19:03.470593 28212 raft_consensus.cc:348] T 0582dc470a2f4985bdbc2cbd8b7e0e4d P 1da32ab00b8748aeb0b9879201a73b6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff2989b8d8bd4cc9ab32b320fbd3d52d" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 45255 } } peers { permanent_uuid: "1da32ab00b8748aeb0b9879201a73b6d" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34529 } } peers { permanent_uuid: "5969fd3075dd4de192f2fc1e163a42b0" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 41963 } }
I20260812 01:19:03.470733 28212 raft_consensus.cc:374] T 0582dc470a2f4985bdbc2cbd8b7e0e4d P 1da32ab00b8748aeb0b9879201a73b6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:19:03.470779 28212 raft_consensus.cc:729] T 0582dc470a2f4985bdbc2cbd8b7e0e4d P 1da32ab00b8748aeb0b9879201a73b6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1da32ab00b8748aeb0b9879201a73b6d, State: Initialized, Role: FOLLOWER
I20260812 01:19:03.470886 28212 consensus_queue.cc:260] T 0582dc470a2f4985bdbc2cbd8b7e0e4d P 1da32ab00b8748aeb0b9879201a73b6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff2989b8d8bd4cc9ab32b320fbd3d52d" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 45255 } } peers { permanent_uuid: "1da32ab00b8748aeb0b9879201a73b6d" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34529 } } peers { permanent_uuid: "5969fd3075dd4de192f2fc1e163a42b0" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 41963 } }
I20260812 01:19:03.471123 28212 ts_tablet_manager.cc:1428] T 0582dc470a2f4985bdbc2cbd8b7e0e4d P 1da32ab00b8748aeb0b9879201a73b6d: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 01:19:03.471191 28180 heartbeater.cc:503] Master 127.24.0.190:43905 was elected leader, sending a full tablet report...
W20260812 01:19:03.503768 27940 tablet.cc:2369] T 0582dc470a2f4985bdbc2cbd8b7e0e4d P 5969fd3075dd4de192f2fc1e163a42b0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 01:19:03.575387 27693 tablet.cc:2369] T 0582dc470a2f4985bdbc2cbd8b7e0e4d P ff2989b8d8bd4cc9ab32b320fbd3d52d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 01:19:03.608950 28222 raft_consensus.cc:482] T 0582dc470a2f4985bdbc2cbd8b7e0e4d P 5969fd3075dd4de192f2fc1e163a42b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 01:19:03.609112 28222 raft_consensus.cc:504] T 0582dc470a2f4985bdbc2cbd8b7e0e4d P 5969fd3075dd4de192f2fc1e163a42b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff2989b8d8bd4cc9ab32b320fbd3d52d" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 45255 } } peers { permanent_uuid: "1da32ab00b8748aeb0b9879201a73b6d" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34529 } } peers { permanent_uuid: "5969fd3075dd4de192f2fc1e163a42b0" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 41963 } }
I20260812 01:19:03.609536 28222 leader_election.cc:288] T 0582dc470a2f4985bdbc2cbd8b7e0e4d P 5969fd3075dd4de192f2fc1e163a42b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ff2989b8d8bd4cc9ab32b320fbd3d52d (127.24.0.129:45255), 1da32ab00b8748aeb0b9879201a73b6d (127.24.0.131:34529)
I20260812 01:19:03.612689 27744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:03.609746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.129:45255 (local address 127.24.0.130:34691)
0812 01:19:03.609806 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:03.609818 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:03.609868 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:03.610046 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:03.610047 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:03.610048 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:19:03.610106 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:03.610108 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:03.610933 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:03.610935 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:03.611726 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:03.611728 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:03.612448 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:03.612449 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:03.612529 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:03.612533 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 01:19:03.612588 (+ 55us) client_negotiation.cc:240] Negotiation successful
0812 01:19:03.612617 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260812 01:19:03.612740 28206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:03.609836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.130:34691 (local address 127.24.0.129:45255)
0812 01:19:03.609882 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:03.609884 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:03.609895 (+ 11us) server_negotiation.cc:407] Connection header received
0812 01:19:03.609928 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:03.609929 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:03.609941 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:03.609993 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:19:03.610201 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:03.610841 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:03.611826 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:03.612368 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:03.612494 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:03.612514 (+ 20us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:03.612600 (+ 86us) server_negotiation.cc:299] Negotiation successful
0812 01:19:03.612650 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 01:19:03.612825 28207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:03.609891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.130:45751 (local address 127.24.0.131:34529)
0812 01:19:03.609957 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:03.609959 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:03.610194 (+ 235us) server_negotiation.cc:407] Connection header received
0812 01:19:03.610263 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:03.610264 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:03.610275 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:03.610314 (+ 39us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:19:03.610470 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:03.610953 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:03.612009 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:03.612438 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:03.612511 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:03.612522 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:03.612718 (+ 196us) server_negotiation.cc:299] Negotiation successful
0812 01:19:03.612766 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 01:19:03.613025 28231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:03.609826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.131:34529 (local address 127.24.0.130:45751)
0812 01:19:03.610080 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:03.610099 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:03.610195 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:03.610333 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:03.610334 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:03.610336 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:19:03.610404 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:03.610406 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:03.611056 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:03.611057 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:03.611918 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:03.611921 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:03.612535 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:03.612536 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:03.612630 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:03.612634 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 01:19:03.612701 (+ 67us) client_negotiation.cc:240] Negotiation successful
0812 01:19:03.612945 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":145,"threads_started":1}
I20260812 01:19:03.613286 27581 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0582dc470a2f4985bdbc2cbd8b7e0e4d" candidate_uuid: "5969fd3075dd4de192f2fc1e163a42b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff2989b8d8bd4cc9ab32b320fbd3d52d" is_pre_election: true
I20260812 01:19:03.613438 27581 raft_consensus.cc:2454] T 0582dc470a2f4985bdbc2cbd8b7e0e4d P ff2989b8d8bd4cc9ab32b320fbd3d52d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5969fd3075dd4de192f2fc1e163a42b0 in term 0.
I20260812 01:19:03.613480 28100 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0582dc470a2f4985bdbc2cbd8b7e0e4d" candidate_uuid: "5969fd3075dd4de192f2fc1e163a42b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1da32ab00b8748aeb0b9879201a73b6d" is_pre_election: true
I20260812 01:19:03.613664 28100 raft_consensus.cc:2454] T 0582dc470a2f4985bdbc2cbd8b7e0e4d P 1da32ab00b8748aeb0b9879201a73b6d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5969fd3075dd4de192f2fc1e163a42b0 in term 0.
I20260812 01:19:03.613703 27732 leader_election.cc:302] T 0582dc470a2f4985bdbc2cbd8b7e0e4d P 5969fd3075dd4de192f2fc1e163a42b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5969fd3075dd4de192f2fc1e163a42b0, ff2989b8d8bd4cc9ab32b320fbd3d52d; no voters:
I20260812 01:19:03.613830 28222 raft_consensus.cc:2781] T 0582dc470a2f4985bdbc2cbd8b7e0e4d P 5969fd3075dd4de192f2fc1e163a42b0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 01:19:03.613919 28222 raft_consensus.cc:482] T 0582dc470a2f4985bdbc2cbd8b7e0e4d P 5969fd3075dd4de192f2fc1e163a42b0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 01:19:03.613957 28222 raft_consensus.cc:3037] T 0582dc470a2f4985bdbc2cbd8b7e0e4d P 5969fd3075dd4de192f2fc1e163a42b0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:19:03.614907 28222 raft_consensus.cc:504] T 0582dc470a2f4985bdbc2cbd8b7e0e4d P 5969fd3075dd4de192f2fc1e163a42b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff2989b8d8bd4cc9ab32b320fbd3d52d" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 45255 } } peers { permanent_uuid: "1da32ab00b8748aeb0b9879201a73b6d" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34529 } } peers { permanent_uuid: "5969fd3075dd4de192f2fc1e163a42b0" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 41963 } }
I20260812 01:19:03.615061 28222 leader_election.cc:288] T 0582dc470a2f4985bdbc2cbd8b7e0e4d P 5969fd3075dd4de192f2fc1e163a42b0 [CANDIDATE]: Term 1 election: Requested vote from peers ff2989b8d8bd4cc9ab32b320fbd3d52d (127.24.0.129:45255), 1da32ab00b8748aeb0b9879201a73b6d (127.24.0.131:34529)
I20260812 01:19:03.615200 28100 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0582dc470a2f4985bdbc2cbd8b7e0e4d" candidate_uuid: "5969fd3075dd4de192f2fc1e163a42b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1da32ab00b8748aeb0b9879201a73b6d"
I20260812 01:19:03.615296 28100 raft_consensus.cc:3037] T 0582dc470a2f4985bdbc2cbd8b7e0e4d P 1da32ab00b8748aeb0b9879201a73b6d [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:19:03.615316 27581 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0582dc470a2f4985bdbc2cbd8b7e0e4d" candidate_uuid: "5969fd3075dd4de192f2fc1e163a42b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff2989b8d8bd4cc9ab32b320fbd3d52d"
I20260812 01:19:03.615388 27581 raft_consensus.cc:3037] T 0582dc470a2f4985bdbc2cbd8b7e0e4d P ff2989b8d8bd4cc9ab32b320fbd3d52d [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:19:03.616276 27581 raft_consensus.cc:2454] T 0582dc470a2f4985bdbc2cbd8b7e0e4d P ff2989b8d8bd4cc9ab32b320fbd3d52d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5969fd3075dd4de192f2fc1e163a42b0 in term 1.
I20260812 01:19:03.616274 28100 raft_consensus.cc:2454] T 0582dc470a2f4985bdbc2cbd8b7e0e4d P 1da32ab00b8748aeb0b9879201a73b6d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5969fd3075dd4de192f2fc1e163a42b0 in term 1.
I20260812 01:19:03.616472 27730 leader_election.cc:302] T 0582dc470a2f4985bdbc2cbd8b7e0e4d P 5969fd3075dd4de192f2fc1e163a42b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1da32ab00b8748aeb0b9879201a73b6d, 5969fd3075dd4de192f2fc1e163a42b0; no voters:
I20260812 01:19:03.616577 28222 raft_consensus.cc:2781] T 0582dc470a2f4985bdbc2cbd8b7e0e4d P 5969fd3075dd4de192f2fc1e163a42b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:19:03.616719 28222 raft_consensus.cc:686] T 0582dc470a2f4985bdbc2cbd8b7e0e4d P 5969fd3075dd4de192f2fc1e163a42b0 [term 1 LEADER]: Becoming Leader. State: Replica: 5969fd3075dd4de192f2fc1e163a42b0, State: Running, Role: LEADER
I20260812 01:19:03.616813 28222 consensus_queue.cc:237] T 0582dc470a2f4985bdbc2cbd8b7e0e4d P 5969fd3075dd4de192f2fc1e163a42b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff2989b8d8bd4cc9ab32b320fbd3d52d" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 45255 } } peers { permanent_uuid: "1da32ab00b8748aeb0b9879201a73b6d" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34529 } } peers { permanent_uuid: "5969fd3075dd4de192f2fc1e163a42b0" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 41963 } }
I20260812 01:19:03.617660 27354 catalog_manager.cc:5495] T 0582dc470a2f4985bdbc2cbd8b7e0e4d P 5969fd3075dd4de192f2fc1e163a42b0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5969fd3075dd4de192f2fc1e163a42b0 (127.24.0.130). New cstate: current_term: 1 leader_uuid: "5969fd3075dd4de192f2fc1e163a42b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff2989b8d8bd4cc9ab32b320fbd3d52d" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 45255 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1da32ab00b8748aeb0b9879201a73b6d" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5969fd3075dd4de192f2fc1e163a42b0" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 41963 } health_report { overall_health: HEALTHY } } }
Aug 12 01:19:03 dist-test-slave-t3q3 krb5kdc[27285](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497542, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.190@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
I20260812 01:19:03.659852 28242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:03.652767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:43905 (local address 127.0.0.1:34766)
0812 01:19:03.652959 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:03.652975 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:03.653070 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:03.653215 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:03.653216 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:03.653573 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:03.653618 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:03.653620 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:03.654273 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:03.654274 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:03.654791 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:03.654793 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:03.655129 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:03.655130 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:03.655189 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:03.655758 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:03.655763 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:03.657380 (+ 1617us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:19:03.658882 (+ 1502us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:03.658889 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:03.658891 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:03.659090 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:03.659268 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:03.659268 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:03.659269 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:03.659348 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:03.659557 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:19:03.659564 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:19:03.659609 (+ 45us) client_negotiation.cc:769] Sending connection context
0812 01:19:03.659663 (+ 54us) client_negotiation.cc:240] Negotiation successful
0812 01:19:03.659754 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":102,"threads_started":1}
I20260812 01:19:03.659901 27435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:03.652836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34766 (local address 127.24.0.190:43905)
0812 01:19:03.652918 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:03.652921 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:03.653049 (+ 128us) server_negotiation.cc:407] Connection header received
0812 01:19:03.653121 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:03.653121 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:03.653132 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:03.653208 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:03.653703 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:03.654188 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:03.654873 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:03.655056 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:03.655131 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:03.657546 (+ 2415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:03.657547 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:03.657548 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:03.657557 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:03.658807 (+ 1250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:03.659156 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:03.659157 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:03.659157 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:03.659209 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:03.659402 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:03.659403 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:03.659403 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:03.659503 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:03.659553 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:03.659694 (+ 141us) server_negotiation.cc:299] Negotiation successful
0812 01:19:03.659802 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":32,"server-negotiator.queue_time_us":53}
I20260812 01:19:03.661862 27354 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.24.0.190@KRBTEST.COM'} at 127.0.0.1:34766:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260812 01:19:03.662070 27354 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 01:19:03.663630 27580 tablet_service.cc:1467] Processing CreateTablet for tablet be0a5b71cde1411c8d26da3aad71a138 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a2b59accb5eb4291bfe4a7c03c370d80]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 01:19:03.663636 27824 tablet_service.cc:1467] Processing CreateTablet for tablet be0a5b71cde1411c8d26da3aad71a138 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a2b59accb5eb4291bfe4a7c03c370d80]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 01:19:03.663719 28059 tablet_service.cc:1467] Processing CreateTablet for tablet be0a5b71cde1411c8d26da3aad71a138 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a2b59accb5eb4291bfe4a7c03c370d80]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 01:19:03.663836 27824 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be0a5b71cde1411c8d26da3aad71a138. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:19:03.663834 27580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be0a5b71cde1411c8d26da3aad71a138. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:19:03.663868 28059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be0a5b71cde1411c8d26da3aad71a138. 1 dirs total, 0 dirs full, 0 dirs failed
W20260812 01:19:03.665164 28181 tablet.cc:2369] T 0582dc470a2f4985bdbc2cbd8b7e0e4d P 1da32ab00b8748aeb0b9879201a73b6d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 01:19:03.665410 28212 tablet_bootstrap.cc:492] T be0a5b71cde1411c8d26da3aad71a138 P 1da32ab00b8748aeb0b9879201a73b6d: Bootstrap starting.
I20260812 01:19:03.665454 28214 tablet_bootstrap.cc:492] T be0a5b71cde1411c8d26da3aad71a138 P ff2989b8d8bd4cc9ab32b320fbd3d52d: Bootstrap starting.
I20260812 01:19:03.665851 28214 tablet_bootstrap.cc:654] T be0a5b71cde1411c8d26da3aad71a138 P ff2989b8d8bd4cc9ab32b320fbd3d52d: Neither blocks nor log segments found. Creating new log.
I20260812 01:19:03.666005 28212 tablet_bootstrap.cc:654] T be0a5b71cde1411c8d26da3aad71a138 P 1da32ab00b8748aeb0b9879201a73b6d: Neither blocks nor log segments found. Creating new log.
I20260812 01:19:03.666221 28213 tablet_bootstrap.cc:492] T be0a5b71cde1411c8d26da3aad71a138 P 5969fd3075dd4de192f2fc1e163a42b0: Bootstrap starting.
I20260812 01:19:03.666708 28212 tablet_bootstrap.cc:492] T be0a5b71cde1411c8d26da3aad71a138 P 1da32ab00b8748aeb0b9879201a73b6d: No bootstrap required, opened a new log
I20260812 01:19:03.666705 28213 tablet_bootstrap.cc:654] T be0a5b71cde1411c8d26da3aad71a138 P 5969fd3075dd4de192f2fc1e163a42b0: Neither blocks nor log segments found. Creating new log.
I20260812 01:19:03.666760 28212 ts_tablet_manager.cc:1397] T be0a5b71cde1411c8d26da3aad71a138 P 1da32ab00b8748aeb0b9879201a73b6d: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 01:19:03.666930 28212 raft_consensus.cc:348] T be0a5b71cde1411c8d26da3aad71a138 P 1da32ab00b8748aeb0b9879201a73b6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff2989b8d8bd4cc9ab32b320fbd3d52d" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 45255 } } peers { permanent_uuid: "5969fd3075dd4de192f2fc1e163a42b0" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 41963 } } peers { permanent_uuid: "1da32ab00b8748aeb0b9879201a73b6d" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34529 } }
I20260812 01:19:03.667009 28212 raft_consensus.cc:374] T be0a5b71cde1411c8d26da3aad71a138 P 1da32ab00b8748aeb0b9879201a73b6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:19:03.667039 28212 raft_consensus.cc:729] T be0a5b71cde1411c8d26da3aad71a138 P 1da32ab00b8748aeb0b9879201a73b6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1da32ab00b8748aeb0b9879201a73b6d, State: Initialized, Role: FOLLOWER
I20260812 01:19:03.667083 28212 consensus_queue.cc:260] T be0a5b71cde1411c8d26da3aad71a138 P 1da32ab00b8748aeb0b9879201a73b6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff2989b8d8bd4cc9ab32b320fbd3d52d" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 45255 } } peers { permanent_uuid: "5969fd3075dd4de192f2fc1e163a42b0" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 41963 } } peers { permanent_uuid: "1da32ab00b8748aeb0b9879201a73b6d" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34529 } }
I20260812 01:19:03.667173 28212 ts_tablet_manager.cc:1428] T be0a5b71cde1411c8d26da3aad71a138 P 1da32ab00b8748aeb0b9879201a73b6d: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 01:19:03.667320 28214 tablet_bootstrap.cc:492] T be0a5b71cde1411c8d26da3aad71a138 P ff2989b8d8bd4cc9ab32b320fbd3d52d: No bootstrap required, opened a new log
I20260812 01:19:03.667364 28214 ts_tablet_manager.cc:1397] T be0a5b71cde1411c8d26da3aad71a138 P ff2989b8d8bd4cc9ab32b320fbd3d52d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 01:19:03.667321 28227 tablet_replica.cc:439] T be0a5b71cde1411c8d26da3aad71a138 P 1da32ab00b8748aeb0b9879201a73b6d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff2989b8d8bd4cc9ab32b320fbd3d52d" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 45255 } } peers { permanent_uuid: "5969fd3075dd4de192f2fc1e163a42b0" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 41963 } } peers { permanent_uuid: "1da32ab00b8748aeb0b9879201a73b6d" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34529 } } }
I20260812 01:19:03.667431 28227 tablet_replica.cc:442] T be0a5b71cde1411c8d26da3aad71a138 P 1da32ab00b8748aeb0b9879201a73b6d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 01:19:03.667528 28214 raft_consensus.cc:348] T be0a5b71cde1411c8d26da3aad71a138 P ff2989b8d8bd4cc9ab32b320fbd3d52d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff2989b8d8bd4cc9ab32b320fbd3d52d" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 45255 } } peers { permanent_uuid: "5969fd3075dd4de192f2fc1e163a42b0" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 41963 } } peers { permanent_uuid: "1da32ab00b8748aeb0b9879201a73b6d" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34529 } }
I20260812 01:19:03.667611 28214 raft_consensus.cc:374] T be0a5b71cde1411c8d26da3aad71a138 P ff2989b8d8bd4cc9ab32b320fbd3d52d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:19:03.667639 28214 raft_consensus.cc:729] T be0a5b71cde1411c8d26da3aad71a138 P ff2989b8d8bd4cc9ab32b320fbd3d52d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff2989b8d8bd4cc9ab32b320fbd3d52d, State: Initialized, Role: FOLLOWER
I20260812 01:19:03.667708 28214 consensus_queue.cc:260] T be0a5b71cde1411c8d26da3aad71a138 P ff2989b8d8bd4cc9ab32b320fbd3d52d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff2989b8d8bd4cc9ab32b320fbd3d52d" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 45255 } } peers { permanent_uuid: "5969fd3075dd4de192f2fc1e163a42b0" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 41963 } } peers { permanent_uuid: "1da32ab00b8748aeb0b9879201a73b6d" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34529 } }
I20260812 01:19:03.667824 28214 ts_tablet_manager.cc:1428] T be0a5b71cde1411c8d26da3aad71a138 P ff2989b8d8bd4cc9ab32b320fbd3d52d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:03.667867 28226 tablet_replica.cc:439] T be0a5b71cde1411c8d26da3aad71a138 P ff2989b8d8bd4cc9ab32b320fbd3d52d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff2989b8d8bd4cc9ab32b320fbd3d52d" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 45255 } } peers { permanent_uuid: "5969fd3075dd4de192f2fc1e163a42b0" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 41963 } } peers { permanent_uuid: "1da32ab00b8748aeb0b9879201a73b6d" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34529 } } }
I20260812 01:19:03.667968 28226 tablet_replica.cc:442] T be0a5b71cde1411c8d26da3aad71a138 P ff2989b8d8bd4cc9ab32b320fbd3d52d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 01:19:03.668094 28213 tablet_bootstrap.cc:492] T be0a5b71cde1411c8d26da3aad71a138 P 5969fd3075dd4de192f2fc1e163a42b0: No bootstrap required, opened a new log
I20260812 01:19:03.668143 28213 ts_tablet_manager.cc:1397] T be0a5b71cde1411c8d26da3aad71a138 P 5969fd3075dd4de192f2fc1e163a42b0: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260812 01:19:03.668291 28213 raft_consensus.cc:348] T be0a5b71cde1411c8d26da3aad71a138 P 5969fd3075dd4de192f2fc1e163a42b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff2989b8d8bd4cc9ab32b320fbd3d52d" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 45255 } } peers { permanent_uuid: "5969fd3075dd4de192f2fc1e163a42b0" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 41963 } } peers { permanent_uuid: "1da32ab00b8748aeb0b9879201a73b6d" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34529 } }
I20260812 01:19:03.668373 28213 raft_consensus.cc:374] T be0a5b71cde1411c8d26da3aad71a138 P 5969fd3075dd4de192f2fc1e163a42b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:19:03.668402 28213 raft_consensus.cc:729] T be0a5b71cde1411c8d26da3aad71a138 P 5969fd3075dd4de192f2fc1e163a42b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5969fd3075dd4de192f2fc1e163a42b0, State: Initialized, Role: FOLLOWER
I20260812 01:19:03.668448 28213 consensus_queue.cc:260] T be0a5b71cde1411c8d26da3aad71a138 P 5969fd3075dd4de192f2fc1e163a42b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff2989b8d8bd4cc9ab32b320fbd3d52d" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 45255 } } peers { permanent_uuid: "5969fd3075dd4de192f2fc1e163a42b0" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 41963 } } peers { permanent_uuid: "1da32ab00b8748aeb0b9879201a73b6d" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34529 } }
I20260812 01:19:03.668537 28213 ts_tablet_manager.cc:1428] T be0a5b71cde1411c8d26da3aad71a138 P 5969fd3075dd4de192f2fc1e163a42b0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:03.668617 28222 tablet_replica.cc:439] T be0a5b71cde1411c8d26da3aad71a138 P 5969fd3075dd4de192f2fc1e163a42b0: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff2989b8d8bd4cc9ab32b320fbd3d52d" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 45255 } } peers { permanent_uuid: "5969fd3075dd4de192f2fc1e163a42b0" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 41963 } } peers { permanent_uuid: "1da32ab00b8748aeb0b9879201a73b6d" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34529 } } }
I20260812 01:19:03.668730 28222 tablet_replica.cc:442] T be0a5b71cde1411c8d26da3aad71a138 P 5969fd3075dd4de192f2fc1e163a42b0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 01:19:03.704546 28222 raft_consensus.cc:482] T be0a5b71cde1411c8d26da3aad71a138 P 5969fd3075dd4de192f2fc1e163a42b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 01:19:03.704646 28222 raft_consensus.cc:504] T be0a5b71cde1411c8d26da3aad71a138 P 5969fd3075dd4de192f2fc1e163a42b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff2989b8d8bd4cc9ab32b320fbd3d52d" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 45255 } } peers { permanent_uuid: "5969fd3075dd4de192f2fc1e163a42b0" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 41963 } } peers { permanent_uuid: "1da32ab00b8748aeb0b9879201a73b6d" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34529 } }
I20260812 01:19:03.704825 28222 leader_election.cc:288] T be0a5b71cde1411c8d26da3aad71a138 P 5969fd3075dd4de192f2fc1e163a42b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ff2989b8d8bd4cc9ab32b320fbd3d52d (127.24.0.129:45255), 1da32ab00b8748aeb0b9879201a73b6d (127.24.0.131:34529)
I20260812 01:19:03.705091 27581 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be0a5b71cde1411c8d26da3aad71a138" candidate_uuid: "5969fd3075dd4de192f2fc1e163a42b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff2989b8d8bd4cc9ab32b320fbd3d52d" is_pre_election: true
I20260812 01:19:03.705089 28100 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be0a5b71cde1411c8d26da3aad71a138" candidate_uuid: "5969fd3075dd4de192f2fc1e163a42b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1da32ab00b8748aeb0b9879201a73b6d" is_pre_election: true
I20260812 01:19:03.705204 28100 raft_consensus.cc:2454] T be0a5b71cde1411c8d26da3aad71a138 P 1da32ab00b8748aeb0b9879201a73b6d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5969fd3075dd4de192f2fc1e163a42b0 in term 0.
I20260812 01:19:03.705209 27581 raft_consensus.cc:2454] T be0a5b71cde1411c8d26da3aad71a138 P ff2989b8d8bd4cc9ab32b320fbd3d52d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5969fd3075dd4de192f2fc1e163a42b0 in term 0.
I20260812 01:19:03.705363 27732 leader_election.cc:302] T be0a5b71cde1411c8d26da3aad71a138 P 5969fd3075dd4de192f2fc1e163a42b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5969fd3075dd4de192f2fc1e163a42b0, ff2989b8d8bd4cc9ab32b320fbd3d52d; no voters:
I20260812 01:19:03.705466 28222 raft_consensus.cc:2781] T be0a5b71cde1411c8d26da3aad71a138 P 5969fd3075dd4de192f2fc1e163a42b0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 01:19:03.705508 28222 raft_consensus.cc:482] T be0a5b71cde1411c8d26da3aad71a138 P 5969fd3075dd4de192f2fc1e163a42b0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 01:19:03.705530 28222 raft_consensus.cc:3037] T be0a5b71cde1411c8d26da3aad71a138 P 5969fd3075dd4de192f2fc1e163a42b0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:19:03.706224 28222 raft_consensus.cc:504] T be0a5b71cde1411c8d26da3aad71a138 P 5969fd3075dd4de192f2fc1e163a42b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff2989b8d8bd4cc9ab32b320fbd3d52d" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 45255 } } peers { permanent_uuid: "5969fd3075dd4de192f2fc1e163a42b0" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 41963 } } peers { permanent_uuid: "1da32ab00b8748aeb0b9879201a73b6d" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34529 } }
I20260812 01:19:03.706368 28222 leader_election.cc:288] T be0a5b71cde1411c8d26da3aad71a138 P 5969fd3075dd4de192f2fc1e163a42b0 [CANDIDATE]: Term 1 election: Requested vote from peers ff2989b8d8bd4cc9ab32b320fbd3d52d (127.24.0.129:45255), 1da32ab00b8748aeb0b9879201a73b6d (127.24.0.131:34529)
I20260812 01:19:03.706542 27581 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be0a5b71cde1411c8d26da3aad71a138" candidate_uuid: "5969fd3075dd4de192f2fc1e163a42b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff2989b8d8bd4cc9ab32b320fbd3d52d"
I20260812 01:19:03.706574 28100 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be0a5b71cde1411c8d26da3aad71a138" candidate_uuid: "5969fd3075dd4de192f2fc1e163a42b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1da32ab00b8748aeb0b9879201a73b6d"
I20260812 01:19:03.706629 27581 raft_consensus.cc:3037] T be0a5b71cde1411c8d26da3aad71a138 P ff2989b8d8bd4cc9ab32b320fbd3d52d [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:19:03.706640 28100 raft_consensus.cc:3037] T be0a5b71cde1411c8d26da3aad71a138 P 1da32ab00b8748aeb0b9879201a73b6d [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:19:03.707410 27581 raft_consensus.cc:2454] T be0a5b71cde1411c8d26da3aad71a138 P ff2989b8d8bd4cc9ab32b320fbd3d52d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5969fd3075dd4de192f2fc1e163a42b0 in term 1.
I20260812 01:19:03.707410 28100 raft_consensus.cc:2454] T be0a5b71cde1411c8d26da3aad71a138 P 1da32ab00b8748aeb0b9879201a73b6d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5969fd3075dd4de192f2fc1e163a42b0 in term 1.
I20260812 01:19:03.707615 27730 leader_election.cc:302] T be0a5b71cde1411c8d26da3aad71a138 P 5969fd3075dd4de192f2fc1e163a42b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1da32ab00b8748aeb0b9879201a73b6d, 5969fd3075dd4de192f2fc1e163a42b0; no voters:
I20260812 01:19:03.707753 28222 raft_consensus.cc:2781] T be0a5b71cde1411c8d26da3aad71a138 P 5969fd3075dd4de192f2fc1e163a42b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:19:03.707816 28222 raft_consensus.cc:686] T be0a5b71cde1411c8d26da3aad71a138 P 5969fd3075dd4de192f2fc1e163a42b0 [term 1 LEADER]: Becoming Leader. State: Replica: 5969fd3075dd4de192f2fc1e163a42b0, State: Running, Role: LEADER
I20260812 01:19:03.707850 28222 consensus_queue.cc:237] T be0a5b71cde1411c8d26da3aad71a138 P 5969fd3075dd4de192f2fc1e163a42b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff2989b8d8bd4cc9ab32b320fbd3d52d" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 45255 } } peers { permanent_uuid: "5969fd3075dd4de192f2fc1e163a42b0" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 41963 } } peers { permanent_uuid: "1da32ab00b8748aeb0b9879201a73b6d" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34529 } }
I20260812 01:19:03.708267 28232 tablet_replica.cc:439] T be0a5b71cde1411c8d26da3aad71a138 P 5969fd3075dd4de192f2fc1e163a42b0: TxnStatusTablet state changed. Reason: New leader 5969fd3075dd4de192f2fc1e163a42b0. Latest consensus state: current_term: 1 leader_uuid: "5969fd3075dd4de192f2fc1e163a42b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff2989b8d8bd4cc9ab32b320fbd3d52d" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 45255 } } peers { permanent_uuid: "5969fd3075dd4de192f2fc1e163a42b0" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 41963 } } peers { permanent_uuid: "1da32ab00b8748aeb0b9879201a73b6d" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34529 } } }
I20260812 01:19:03.708369 28232 tablet_replica.cc:442] T be0a5b71cde1411c8d26da3aad71a138 P 5969fd3075dd4de192f2fc1e163a42b0: This TxnStatusTablet replica's current role is: LEADER
I20260812 01:19:03.708566 27354 catalog_manager.cc:5495] T be0a5b71cde1411c8d26da3aad71a138 P 5969fd3075dd4de192f2fc1e163a42b0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5969fd3075dd4de192f2fc1e163a42b0 (127.24.0.130). New cstate: current_term: 1 leader_uuid: "5969fd3075dd4de192f2fc1e163a42b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff2989b8d8bd4cc9ab32b320fbd3d52d" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 45255 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5969fd3075dd4de192f2fc1e163a42b0" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 41963 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1da32ab00b8748aeb0b9879201a73b6d" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34529 } health_report { overall_health: UNKNOWN } } }
I20260812 01:19:03.708637 28247 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260812 01:19:03.708783 28247 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260812 01:19:03.718461 28242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:03.715517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.130:41963 (local address 127.0.0.1:49630)
0812 01:19:03.715585 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:03.715597 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:03.715649 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:03.715825 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:03.715826 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:03.715828 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 01:19:03.715875 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:03.715876 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:03.716781 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:03.716782 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:03.717424 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:03.717426 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:03.717902 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:03.717902 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:03.717952 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:03.717968 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 01:19:03.718259 (+ 291us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 01:19:03.718269 (+ 10us) client_negotiation.cc:769] Sending connection context
0812 01:19:03.718329 (+ 60us) client_negotiation.cc:240] Negotiation successful
0812 01:19:03.718383 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260812 01:19:03.718680 28211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:03.715571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49630 (local address 127.24.0.130:41963)
0812 01:19:03.715619 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:03.715621 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:03.715650 (+ 29us) server_negotiation.cc:407] Connection header received
0812 01:19:03.715716 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:03.715717 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:03.715738 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:03.715815 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:19:03.715922 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:03.716703 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:03.717509 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:03.717827 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:03.717902 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:03.718076 (+ 174us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:19:03.718191 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:19:03.718291 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:03.718576 (+ 285us) server_negotiation.cc:299] Negotiation successful
0812 01:19:03.718620 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 01:19:03.720507 27581 raft_consensus.cc:1261] T be0a5b71cde1411c8d26da3aad71a138 P ff2989b8d8bd4cc9ab32b320fbd3d52d [term 1 FOLLOWER]: Refusing update from remote peer 5969fd3075dd4de192f2fc1e163a42b0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:19:03.720532 28100 raft_consensus.cc:1261] T be0a5b71cde1411c8d26da3aad71a138 P 1da32ab00b8748aeb0b9879201a73b6d [term 1 FOLLOWER]: Refusing update from remote peer 5969fd3075dd4de192f2fc1e163a42b0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:19:03.720876 28232 consensus_queue.cc:1035] T be0a5b71cde1411c8d26da3aad71a138 P 5969fd3075dd4de192f2fc1e163a42b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1da32ab00b8748aeb0b9879201a73b6d" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:19:03.721019 28222 consensus_queue.cc:1035] T be0a5b71cde1411c8d26da3aad71a138 P 5969fd3075dd4de192f2fc1e163a42b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff2989b8d8bd4cc9ab32b320fbd3d52d" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 45255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:19:03.722184 28226 tablet_replica.cc:439] T be0a5b71cde1411c8d26da3aad71a138 P ff2989b8d8bd4cc9ab32b320fbd3d52d: TxnStatusTablet state changed. Reason: New leader 5969fd3075dd4de192f2fc1e163a42b0. Latest consensus state: current_term: 1 leader_uuid: "5969fd3075dd4de192f2fc1e163a42b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff2989b8d8bd4cc9ab32b320fbd3d52d" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 45255 } } peers { permanent_uuid: "5969fd3075dd4de192f2fc1e163a42b0" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 41963 } } peers { permanent_uuid: "1da32ab00b8748aeb0b9879201a73b6d" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34529 } } }
I20260812 01:19:03.722285 28226 tablet_replica.cc:442] T be0a5b71cde1411c8d26da3aad71a138 P ff2989b8d8bd4cc9ab32b320fbd3d52d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 01:19:03.722272 28227 tablet_replica.cc:439] T be0a5b71cde1411c8d26da3aad71a138 P 1da32ab00b8748aeb0b9879201a73b6d: TxnStatusTablet state changed. Reason: New leader 5969fd3075dd4de192f2fc1e163a42b0. Latest consensus state: current_term: 1 leader_uuid: "5969fd3075dd4de192f2fc1e163a42b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff2989b8d8bd4cc9ab32b320fbd3d52d" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 45255 } } peers { permanent_uuid: "5969fd3075dd4de192f2fc1e163a42b0" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 41963 } } peers { permanent_uuid: "1da32ab00b8748aeb0b9879201a73b6d" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34529 } } }
I20260812 01:19:03.722370 28227 tablet_replica.cc:442] T be0a5b71cde1411c8d26da3aad71a138 P 1da32ab00b8748aeb0b9879201a73b6d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 01:19:03.723352 28222 tablet_replica.cc:439] T be0a5b71cde1411c8d26da3aad71a138 P 5969fd3075dd4de192f2fc1e163a42b0: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5969fd3075dd4de192f2fc1e163a42b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff2989b8d8bd4cc9ab32b320fbd3d52d" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 45255 } } peers { permanent_uuid: "5969fd3075dd4de192f2fc1e163a42b0" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 41963 } } peers { permanent_uuid: "1da32ab00b8748aeb0b9879201a73b6d" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34529 } } }
I20260812 01:19:03.723444 28222 tablet_replica.cc:442] T be0a5b71cde1411c8d26da3aad71a138 P 5969fd3075dd4de192f2fc1e163a42b0: This TxnStatusTablet replica's current role is: LEADER
I20260812 01:19:03.723629 28227 tablet_replica.cc:439] T be0a5b71cde1411c8d26da3aad71a138 P 1da32ab00b8748aeb0b9879201a73b6d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5969fd3075dd4de192f2fc1e163a42b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff2989b8d8bd4cc9ab32b320fbd3d52d" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 45255 } } peers { permanent_uuid: "5969fd3075dd4de192f2fc1e163a42b0" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 41963 } } peers { permanent_uuid: "1da32ab00b8748aeb0b9879201a73b6d" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34529 } } }
I20260812 01:19:03.723709 28227 tablet_replica.cc:442] T be0a5b71cde1411c8d26da3aad71a138 P 1da32ab00b8748aeb0b9879201a73b6d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 01:19:03.724244 28222 tablet_replica.cc:439] T be0a5b71cde1411c8d26da3aad71a138 P 5969fd3075dd4de192f2fc1e163a42b0: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5969fd3075dd4de192f2fc1e163a42b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff2989b8d8bd4cc9ab32b320fbd3d52d" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 45255 } } peers { permanent_uuid: "5969fd3075dd4de192f2fc1e163a42b0" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 41963 } } peers { permanent_uuid: "1da32ab00b8748aeb0b9879201a73b6d" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34529 } } }
I20260812 01:19:03.724323 28222 tablet_replica.cc:442] T be0a5b71cde1411c8d26da3aad71a138 P 5969fd3075dd4de192f2fc1e163a42b0: This TxnStatusTablet replica's current role is: LEADER
I20260812 01:19:03.724790 28226 tablet_replica.cc:439] T be0a5b71cde1411c8d26da3aad71a138 P ff2989b8d8bd4cc9ab32b320fbd3d52d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5969fd3075dd4de192f2fc1e163a42b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff2989b8d8bd4cc9ab32b320fbd3d52d" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 45255 } } peers { permanent_uuid: "5969fd3075dd4de192f2fc1e163a42b0" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 41963 } } peers { permanent_uuid: "1da32ab00b8748aeb0b9879201a73b6d" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34529 } } }
I20260812 01:19:03.724879 28226 tablet_replica.cc:442] T be0a5b71cde1411c8d26da3aad71a138 P ff2989b8d8bd4cc9ab32b320fbd3d52d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 01:19:03.732501 28211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:03.729980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49640 (local address 127.24.0.130:41963)
0812 01:19:03.730017 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:03.730018 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:03.730062 (+ 44us) server_negotiation.cc:407] Connection header received
0812 01:19:03.730130 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:03.730131 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:03.730139 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:03.730185 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:19:03.730306 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:03.730740 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:03.731832 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:03.732001 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:03.732050 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:03.732275 (+ 225us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:19:03.732307 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:19:03.732356 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:03.732437 (+ 81us) server_negotiation.cc:299] Negotiation successful
0812 01:19:03.732454 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260812 01:19:03.737393 28211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:03.735122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.130:55451 (local address 127.24.0.130:41963)
0812 01:19:03.735176 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:03.735178 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:03.735225 (+ 47us) server_negotiation.cc:407] Connection header received
0812 01:19:03.735294 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:03.735295 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:03.735305 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:03.735370 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:19:03.735505 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:03.735934 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:03.736719 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:03.736889 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:03.736960 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:03.737148 (+ 188us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:19:03.737184 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:19:03.737245 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:03.737306 (+ 61us) server_negotiation.cc:299] Negotiation successful
0812 01:19:03.737319 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 01:19:03.737408 27953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:03.735063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.130:41963 (local address 127.24.0.130:55451)
0812 01:19:03.735182 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:03.735192 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:03.735240 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:03.735372 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:03.735373 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:03.735374 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 01:19:03.735430 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:03.735431 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:03.736018 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:03.736020 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:03.736619 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:03.736621 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:03.736961 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:03.736962 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:03.737019 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:03.737036 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 01:19:03.737246 (+ 210us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 01:19:03.737248 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 01:19:03.737300 (+ 52us) client_negotiation.cc:240] Negotiation successful
0812 01:19:03.737336 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69}
I20260812 01:19:03.739679 27581 raft_consensus.cc:1261] T 0582dc470a2f4985bdbc2cbd8b7e0e4d P ff2989b8d8bd4cc9ab32b320fbd3d52d [term 1 FOLLOWER]: Refusing update from remote peer 5969fd3075dd4de192f2fc1e163a42b0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:19:03.739893 28100 raft_consensus.cc:1261] T 0582dc470a2f4985bdbc2cbd8b7e0e4d P 1da32ab00b8748aeb0b9879201a73b6d [term 1 FOLLOWER]: Refusing update from remote peer 5969fd3075dd4de192f2fc1e163a42b0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:19:03.739890 28222 consensus_queue.cc:1035] T 0582dc470a2f4985bdbc2cbd8b7e0e4d P 5969fd3075dd4de192f2fc1e163a42b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff2989b8d8bd4cc9ab32b320fbd3d52d" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 45255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:19:03.740119 28222 consensus_queue.cc:1035] T 0582dc470a2f4985bdbc2cbd8b7e0e4d P 5969fd3075dd4de192f2fc1e163a42b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1da32ab00b8748aeb0b9879201a73b6d" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:19:03.741467 28252 mvcc.cc:205] Tried to move back new op lower bound from 7317493939156582400 to 7317493938655088640. Current Snapshot: MvccSnapshot[applied={T|T < 7317493939156582400}]
I20260812 01:19:03.741663 28255 mvcc.cc:205] Tried to move back new op lower bound from 7317493939156582400 to 7317493938655088640. Current Snapshot: MvccSnapshot[applied={T|T < 7317493939156582400}]
I20260812 01:19:03.742309 28253 mvcc.cc:205] Tried to move back new op lower bound from 7317493939156582400 to 7317493938655088640. Current Snapshot: MvccSnapshot[applied={T|T < 7317493939156582400}]
I20260812 01:19:03.754488 27354 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34760:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:19:03.754601 27354 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34760:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:19:03.755497 27354 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d3cf79f36f094ec0aee5f37e3cf9fe7c: Sending DeleteTablet for 3 replicas of tablet 0582dc470a2f4985bdbc2cbd8b7e0e4d
I20260812 01:19:03.755806 27580 tablet_service.cc:1514] Processing DeleteTablet for tablet 0582dc470a2f4985bdbc2cbd8b7e0e4d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:19:03 UTC) from {username='kudu', principal='kudu/127.24.0.190@KRBTEST.COM'} at 127.0.0.1:35958
I20260812 01:19:03.755877 27824 tablet_service.cc:1514] Processing DeleteTablet for tablet 0582dc470a2f4985bdbc2cbd8b7e0e4d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:19:03 UTC) from {username='kudu', principal='kudu/127.24.0.190@KRBTEST.COM'} at 127.0.0.1:49626
I20260812 01:19:03.756017 28059 tablet_service.cc:1514] Processing DeleteTablet for tablet 0582dc470a2f4985bdbc2cbd8b7e0e4d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:19:03 UTC) from {username='kudu', principal='kudu/127.24.0.190@KRBTEST.COM'} at 127.0.0.1:39502
I20260812 01:19:03.756145 27354 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34760:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260812 01:19:03.756253 28276 tablet_replica.cc:330] T 0582dc470a2f4985bdbc2cbd8b7e0e4d P 1da32ab00b8748aeb0b9879201a73b6d: stopping tablet replica
W20260812 01:19:03.756311 27354 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 01:19:03.756346 28276 raft_consensus.cc:2229] T 0582dc470a2f4985bdbc2cbd8b7e0e4d P 1da32ab00b8748aeb0b9879201a73b6d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 01:19:03.756402 28276 raft_consensus.cc:2258] T 0582dc470a2f4985bdbc2cbd8b7e0e4d P 1da32ab00b8748aeb0b9879201a73b6d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:19:03.756606 28274 tablet_replica.cc:330] T 0582dc470a2f4985bdbc2cbd8b7e0e4d P 5969fd3075dd4de192f2fc1e163a42b0: stopping tablet replica
I20260812 01:19:03.756716 28274 raft_consensus.cc:2229] T 0582dc470a2f4985bdbc2cbd8b7e0e4d P 5969fd3075dd4de192f2fc1e163a42b0 [term 1 LEADER]: Raft consensus shutting down.
I20260812 01:19:03.756929 28273 tablet_replica.cc:330] T 0582dc470a2f4985bdbc2cbd8b7e0e4d P ff2989b8d8bd4cc9ab32b320fbd3d52d: stopping tablet replica
I20260812 01:19:03.757032 28273 raft_consensus.cc:2229] T 0582dc470a2f4985bdbc2cbd8b7e0e4d P ff2989b8d8bd4cc9ab32b320fbd3d52d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 01:19:03.757081 28273 raft_consensus.cc:2258] T 0582dc470a2f4985bdbc2cbd8b7e0e4d P ff2989b8d8bd4cc9ab32b320fbd3d52d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:19:03.757496 28274 raft_consensus.cc:2258] T 0582dc470a2f4985bdbc2cbd8b7e0e4d P 5969fd3075dd4de192f2fc1e163a42b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:19:03.757490 27580 tablet_service.cc:1467] Processing CreateTablet for tablet d036f772869f43cf9880eb8a2230faf4 (DEFAULT_TABLE table=test-table [id=54b3071f313f4678a3b3bbec44922a69]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:19:03.757570 28059 tablet_service.cc:1467] Processing CreateTablet for tablet d036f772869f43cf9880eb8a2230faf4 (DEFAULT_TABLE table=test-table [id=54b3071f313f4678a3b3bbec44922a69]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:19:03.757637 27580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d036f772869f43cf9880eb8a2230faf4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:19:03.757876 28059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d036f772869f43cf9880eb8a2230faf4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:19:03.757948 28274 ts_tablet_manager.cc:1905] T 0582dc470a2f4985bdbc2cbd8b7e0e4d P 5969fd3075dd4de192f2fc1e163a42b0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:19:03.759065 28214 tablet_bootstrap.cc:492] T d036f772869f43cf9880eb8a2230faf4 P ff2989b8d8bd4cc9ab32b320fbd3d52d: Bootstrap starting.
I20260812 01:19:03.759248 27824 tablet_service.cc:1467] Processing CreateTablet for tablet d036f772869f43cf9880eb8a2230faf4 (DEFAULT_TABLE table=test-table [id=54b3071f313f4678a3b3bbec44922a69]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:19:03.759310 28212 tablet_bootstrap.cc:492] T d036f772869f43cf9880eb8a2230faf4 P 1da32ab00b8748aeb0b9879201a73b6d: Bootstrap starting.
I20260812 01:19:03.759390 27824 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d036f772869f43cf9880eb8a2230faf4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:19:03.759519 28214 tablet_bootstrap.cc:654] T d036f772869f43cf9880eb8a2230faf4 P ff2989b8d8bd4cc9ab32b320fbd3d52d: Neither blocks nor log segments found. Creating new log.
I20260812 01:19:03.759706 28212 tablet_bootstrap.cc:654] T d036f772869f43cf9880eb8a2230faf4 P 1da32ab00b8748aeb0b9879201a73b6d: Neither blocks nor log segments found. Creating new log.
I20260812 01:19:03.760138 28274 ts_tablet_manager.cc:1918] T 0582dc470a2f4985bdbc2cbd8b7e0e4d P 5969fd3075dd4de192f2fc1e163a42b0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 01:19:03.760210 28274 log.cc:1196] T 0582dc470a2f4985bdbc2cbd8b7e0e4d P 5969fd3075dd4de192f2fc1e163a42b0: Deleting WAL directory at /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/minicluster-data/ts-1/wal/wals/0582dc470a2f4985bdbc2cbd8b7e0e4d
I20260812 01:19:03.760475 28274 ts_tablet_manager.cc:1939] T 0582dc470a2f4985bdbc2cbd8b7e0e4d P 5969fd3075dd4de192f2fc1e163a42b0: Deleting consensus metadata
I20260812 01:19:03.760526 28276 ts_tablet_manager.cc:1905] T 0582dc470a2f4985bdbc2cbd8b7e0e4d P 1da32ab00b8748aeb0b9879201a73b6d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:19:03.760705 28213 tablet_bootstrap.cc:492] T d036f772869f43cf9880eb8a2230faf4 P 5969fd3075dd4de192f2fc1e163a42b0: Bootstrap starting.
I20260812 01:19:03.761237 28213 tablet_bootstrap.cc:654] T d036f772869f43cf9880eb8a2230faf4 P 5969fd3075dd4de192f2fc1e163a42b0: Neither blocks nor log segments found. Creating new log.
I20260812 01:19:03.761301 27329 catalog_manager.cc:4841] TS 5969fd3075dd4de192f2fc1e163a42b0 (127.24.0.130:41963): tablet 0582dc470a2f4985bdbc2cbd8b7e0e4d (table test-table [id=b7a7087ccd054a429dc7a988aa2923d3]) successfully deleted
I20260812 01:19:03.761893 28213 tablet_bootstrap.cc:492] T d036f772869f43cf9880eb8a2230faf4 P 5969fd3075dd4de192f2fc1e163a42b0: No bootstrap required, opened a new log
I20260812 01:19:03.761956 28213 ts_tablet_manager.cc:1397] T d036f772869f43cf9880eb8a2230faf4 P 5969fd3075dd4de192f2fc1e163a42b0: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 01:19:03.762116 28213 raft_consensus.cc:348] T d036f772869f43cf9880eb8a2230faf4 P 5969fd3075dd4de192f2fc1e163a42b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff2989b8d8bd4cc9ab32b320fbd3d52d" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 45255 } } peers { permanent_uuid: "5969fd3075dd4de192f2fc1e163a42b0" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 41963 } } peers { permanent_uuid: "1da32ab00b8748aeb0b9879201a73b6d" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34529 } }
I20260812 01:19:03.762192 28213 raft_consensus.cc:374] T d036f772869f43cf9880eb8a2230faf4 P 5969fd3075dd4de192f2fc1e163a42b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:19:03.762218 28213 raft_consensus.cc:729] T d036f772869f43cf9880eb8a2230faf4 P 5969fd3075dd4de192f2fc1e163a42b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5969fd3075dd4de192f2fc1e163a42b0, State: Initialized, Role: FOLLOWER
I20260812 01:19:03.762266 28213 consensus_queue.cc:260] T d036f772869f43cf9880eb8a2230faf4 P 5969fd3075dd4de192f2fc1e163a42b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff2989b8d8bd4cc9ab32b320fbd3d52d" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 45255 } } peers { permanent_uuid: "5969fd3075dd4de192f2fc1e163a42b0" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 41963 } } peers { permanent_uuid: "1da32ab00b8748aeb0b9879201a73b6d" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34529 } }
I20260812 01:19:03.762380 28213 ts_tablet_manager.cc:1428] T d036f772869f43cf9880eb8a2230faf4 P 5969fd3075dd4de192f2fc1e163a42b0: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 01:19:03.762588 28273 ts_tablet_manager.cc:1905] T 0582dc470a2f4985bdbc2cbd8b7e0e4d P ff2989b8d8bd4cc9ab32b320fbd3d52d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:19:03.764127 28276 ts_tablet_manager.cc:1918] T 0582dc470a2f4985bdbc2cbd8b7e0e4d P 1da32ab00b8748aeb0b9879201a73b6d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 01:19:03.764242 28276 log.cc:1196] T 0582dc470a2f4985bdbc2cbd8b7e0e4d P 1da32ab00b8748aeb0b9879201a73b6d: Deleting WAL directory at /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/minicluster-data/ts-2/wal/wals/0582dc470a2f4985bdbc2cbd8b7e0e4d
I20260812 01:19:03.764249 28273 ts_tablet_manager.cc:1918] T 0582dc470a2f4985bdbc2cbd8b7e0e4d P ff2989b8d8bd4cc9ab32b320fbd3d52d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 01:19:03.764317 28273 log.cc:1196] T 0582dc470a2f4985bdbc2cbd8b7e0e4d P ff2989b8d8bd4cc9ab32b320fbd3d52d: Deleting WAL directory at /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/minicluster-data/ts-0/wal/wals/0582dc470a2f4985bdbc2cbd8b7e0e4d
I20260812 01:19:03.764511 28276 ts_tablet_manager.cc:1939] T 0582dc470a2f4985bdbc2cbd8b7e0e4d P 1da32ab00b8748aeb0b9879201a73b6d: Deleting consensus metadata
I20260812 01:19:03.764636 28273 ts_tablet_manager.cc:1939] T 0582dc470a2f4985bdbc2cbd8b7e0e4d P ff2989b8d8bd4cc9ab32b320fbd3d52d: Deleting consensus metadata
I20260812 01:19:03.764976 27329 catalog_manager.cc:4841] TS 1da32ab00b8748aeb0b9879201a73b6d (127.24.0.131:34529): tablet 0582dc470a2f4985bdbc2cbd8b7e0e4d (table test-table [id=b7a7087ccd054a429dc7a988aa2923d3]) successfully deleted
I20260812 01:19:03.765033 27332 catalog_manager.cc:4841] TS ff2989b8d8bd4cc9ab32b320fbd3d52d (127.24.0.129:45255): tablet 0582dc470a2f4985bdbc2cbd8b7e0e4d (table test-table [id=b7a7087ccd054a429dc7a988aa2923d3]) successfully deleted
I20260812 01:19:03.768471 28212 tablet_bootstrap.cc:492] T d036f772869f43cf9880eb8a2230faf4 P 1da32ab00b8748aeb0b9879201a73b6d: No bootstrap required, opened a new log
I20260812 01:19:03.768543 28212 ts_tablet_manager.cc:1397] T d036f772869f43cf9880eb8a2230faf4 P 1da32ab00b8748aeb0b9879201a73b6d: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.000s
I20260812 01:19:03.768687 28212 raft_consensus.cc:348] T d036f772869f43cf9880eb8a2230faf4 P 1da32ab00b8748aeb0b9879201a73b6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff2989b8d8bd4cc9ab32b320fbd3d52d" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 45255 } } peers { permanent_uuid: "5969fd3075dd4de192f2fc1e163a42b0" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 41963 } } peers { permanent_uuid: "1da32ab00b8748aeb0b9879201a73b6d" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34529 } }
I20260812 01:19:03.768769 28212 raft_consensus.cc:374] T d036f772869f43cf9880eb8a2230faf4 P 1da32ab00b8748aeb0b9879201a73b6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:19:03.768803 28212 raft_consensus.cc:729] T d036f772869f43cf9880eb8a2230faf4 P 1da32ab00b8748aeb0b9879201a73b6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1da32ab00b8748aeb0b9879201a73b6d, State: Initialized, Role: FOLLOWER
I20260812 01:19:03.768857 28212 consensus_queue.cc:260] T d036f772869f43cf9880eb8a2230faf4 P 1da32ab00b8748aeb0b9879201a73b6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff2989b8d8bd4cc9ab32b320fbd3d52d" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 45255 } } peers { permanent_uuid: "5969fd3075dd4de192f2fc1e163a42b0" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 41963 } } peers { permanent_uuid: "1da32ab00b8748aeb0b9879201a73b6d" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34529 } }
I20260812 01:19:03.768950 28212 ts_tablet_manager.cc:1428] T d036f772869f43cf9880eb8a2230faf4 P 1da32ab00b8748aeb0b9879201a73b6d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:03.769156 28214 tablet_bootstrap.cc:492] T d036f772869f43cf9880eb8a2230faf4 P ff2989b8d8bd4cc9ab32b320fbd3d52d: No bootstrap required, opened a new log
I20260812 01:19:03.769207 28214 ts_tablet_manager.cc:1397] T d036f772869f43cf9880eb8a2230faf4 P ff2989b8d8bd4cc9ab32b320fbd3d52d: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.000s
I20260812 01:19:03.769358 28214 raft_consensus.cc:348] T d036f772869f43cf9880eb8a2230faf4 P ff2989b8d8bd4cc9ab32b320fbd3d52d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff2989b8d8bd4cc9ab32b320fbd3d52d" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 45255 } } peers { permanent_uuid: "5969fd3075dd4de192f2fc1e163a42b0" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 41963 } } peers { permanent_uuid: "1da32ab00b8748aeb0b9879201a73b6d" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34529 } }
I20260812 01:19:03.769438 28214 raft_consensus.cc:374] T d036f772869f43cf9880eb8a2230faf4 P ff2989b8d8bd4cc9ab32b320fbd3d52d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:19:03.769464 28214 raft_consensus.cc:729] T d036f772869f43cf9880eb8a2230faf4 P ff2989b8d8bd4cc9ab32b320fbd3d52d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff2989b8d8bd4cc9ab32b320fbd3d52d, State: Initialized, Role: FOLLOWER
I20260812 01:19:03.769511 28214 consensus_queue.cc:260] T d036f772869f43cf9880eb8a2230faf4 P ff2989b8d8bd4cc9ab32b320fbd3d52d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff2989b8d8bd4cc9ab32b320fbd3d52d" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 45255 } } peers { permanent_uuid: "5969fd3075dd4de192f2fc1e163a42b0" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 41963 } } peers { permanent_uuid: "1da32ab00b8748aeb0b9879201a73b6d" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34529 } }
I20260812 01:19:03.769593 28214 ts_tablet_manager.cc:1428] T d036f772869f43cf9880eb8a2230faf4 P ff2989b8d8bd4cc9ab32b320fbd3d52d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:03.821820 28226 raft_consensus.cc:482] T d036f772869f43cf9880eb8a2230faf4 P ff2989b8d8bd4cc9ab32b320fbd3d52d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 01:19:03.821947 28226 raft_consensus.cc:504] T d036f772869f43cf9880eb8a2230faf4 P ff2989b8d8bd4cc9ab32b320fbd3d52d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff2989b8d8bd4cc9ab32b320fbd3d52d" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 45255 } } peers { permanent_uuid: "5969fd3075dd4de192f2fc1e163a42b0" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 41963 } } peers { permanent_uuid: "1da32ab00b8748aeb0b9879201a73b6d" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34529 } }
I20260812 01:19:03.822296 28226 leader_election.cc:288] T d036f772869f43cf9880eb8a2230faf4 P ff2989b8d8bd4cc9ab32b320fbd3d52d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5969fd3075dd4de192f2fc1e163a42b0 (127.24.0.130:41963), 1da32ab00b8748aeb0b9879201a73b6d (127.24.0.131:34529)
I20260812 01:19:03.825989 28282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:03.822493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.130:41963 (local address 127.24.0.129:34469)
0812 01:19:03.822630 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:03.822642 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:03.822758 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:03.822978 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:03.822979 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:03.822981 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:19:03.823059 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:03.823061 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:03.823699 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:03.823700 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:03.824913 (+ 1213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:03.824916 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:03.825695 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:03.825703 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:03.825779 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:03.825783 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 01:19:03.825843 (+ 60us) client_negotiation.cc:240] Negotiation successful
0812 01:19:03.825896 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":75,"threads_started":1}
I20260812 01:19:03.825985 28211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:03.822556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.129:34469 (local address 127.24.0.130:41963)
0812 01:19:03.822630 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:03.822632 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:03.822725 (+ 93us) server_negotiation.cc:407] Connection header received
0812 01:19:03.822870 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:03.822870 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:03.822881 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:03.822959 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:19:03.823125 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:03.823610 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:03.825022 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:03.825587 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:03.825679 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:03.825692 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:03.825866 (+ 174us) server_negotiation.cc:299] Negotiation successful
0812 01:19:03.825911 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 01:19:03.826325 27861 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d036f772869f43cf9880eb8a2230faf4" candidate_uuid: "ff2989b8d8bd4cc9ab32b320fbd3d52d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5969fd3075dd4de192f2fc1e163a42b0" is_pre_election: true
I20260812 01:19:03.826469 27861 raft_consensus.cc:2454] T d036f772869f43cf9880eb8a2230faf4 P 5969fd3075dd4de192f2fc1e163a42b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff2989b8d8bd4cc9ab32b320fbd3d52d in term 0.
I20260812 01:19:03.826571 28207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:03.822746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.129:35289 (local address 127.24.0.131:34529)
0812 01:19:03.822792 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:03.822794 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:03.822854 (+ 60us) server_negotiation.cc:407] Connection header received
0812 01:19:03.822961 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:03.822962 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:03.822971 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:03.823020 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:19:03.823190 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:03.823960 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:03.825256 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:03.825849 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:03.825909 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:03.825917 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:03.826455 (+ 538us) server_negotiation.cc:299] Negotiation successful
0812 01:19:03.826504 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 01:19:03.826598 28283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:03.822679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.131:34529 (local address 127.24.0.129:35289)
0812 01:19:03.822788 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:03.822798 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:03.822886 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:03.823039 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:03.823040 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:03.823043 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:19:03.823123 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:03.823124 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:03.824166 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:03.824167 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:03.825160 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:03.825162 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:03.826301 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:03.826302 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:03.826376 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:03.826380 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 01:19:03.826464 (+ 84us) client_negotiation.cc:240] Negotiation successful
0812 01:19:03.826501 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":36,"threads_started":1}
I20260812 01:19:03.826699 27504 leader_election.cc:302] T d036f772869f43cf9880eb8a2230faf4 P ff2989b8d8bd4cc9ab32b320fbd3d52d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5969fd3075dd4de192f2fc1e163a42b0, ff2989b8d8bd4cc9ab32b320fbd3d52d; no voters:
I20260812 01:19:03.826841 28226 raft_consensus.cc:2781] T d036f772869f43cf9880eb8a2230faf4 P ff2989b8d8bd4cc9ab32b320fbd3d52d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 01:19:03.826879 28226 raft_consensus.cc:482] T d036f772869f43cf9880eb8a2230faf4 P ff2989b8d8bd4cc9ab32b320fbd3d52d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 01:19:03.826898 28226 raft_consensus.cc:3037] T d036f772869f43cf9880eb8a2230faf4 P ff2989b8d8bd4cc9ab32b320fbd3d52d [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:19:03.827013 28100 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d036f772869f43cf9880eb8a2230faf4" candidate_uuid: "ff2989b8d8bd4cc9ab32b320fbd3d52d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1da32ab00b8748aeb0b9879201a73b6d" is_pre_election: true
I20260812 01:19:03.827095 28100 raft_consensus.cc:2454] T d036f772869f43cf9880eb8a2230faf4 P 1da32ab00b8748aeb0b9879201a73b6d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff2989b8d8bd4cc9ab32b320fbd3d52d in term 0.
I20260812 01:19:03.827718 28226 raft_consensus.cc:504] T d036f772869f43cf9880eb8a2230faf4 P ff2989b8d8bd4cc9ab32b320fbd3d52d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff2989b8d8bd4cc9ab32b320fbd3d52d" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 45255 } } peers { permanent_uuid: "5969fd3075dd4de192f2fc1e163a42b0" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 41963 } } peers { permanent_uuid: "1da32ab00b8748aeb0b9879201a73b6d" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34529 } }
I20260812 01:19:03.827850 28226 leader_election.cc:288] T d036f772869f43cf9880eb8a2230faf4 P ff2989b8d8bd4cc9ab32b320fbd3d52d [CANDIDATE]: Term 1 election: Requested vote from peers 5969fd3075dd4de192f2fc1e163a42b0 (127.24.0.130:41963), 1da32ab00b8748aeb0b9879201a73b6d (127.24.0.131:34529)
I20260812 01:19:03.828047 27861 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d036f772869f43cf9880eb8a2230faf4" candidate_uuid: "ff2989b8d8bd4cc9ab32b320fbd3d52d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5969fd3075dd4de192f2fc1e163a42b0"
I20260812 01:19:03.828047 28100 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d036f772869f43cf9880eb8a2230faf4" candidate_uuid: "ff2989b8d8bd4cc9ab32b320fbd3d52d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1da32ab00b8748aeb0b9879201a73b6d"
I20260812 01:19:03.828150 27861 raft_consensus.cc:3037] T d036f772869f43cf9880eb8a2230faf4 P 5969fd3075dd4de192f2fc1e163a42b0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:19:03.828150 28100 raft_consensus.cc:3037] T d036f772869f43cf9880eb8a2230faf4 P 1da32ab00b8748aeb0b9879201a73b6d [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:19:03.828866 28100 raft_consensus.cc:2454] T d036f772869f43cf9880eb8a2230faf4 P 1da32ab00b8748aeb0b9879201a73b6d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff2989b8d8bd4cc9ab32b320fbd3d52d in term 1.
I20260812 01:19:03.829037 27504 leader_election.cc:302] T d036f772869f43cf9880eb8a2230faf4 P ff2989b8d8bd4cc9ab32b320fbd3d52d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1da32ab00b8748aeb0b9879201a73b6d, ff2989b8d8bd4cc9ab32b320fbd3d52d; no voters:
I20260812 01:19:03.829080 27861 raft_consensus.cc:2454] T d036f772869f43cf9880eb8a2230faf4 P 5969fd3075dd4de192f2fc1e163a42b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff2989b8d8bd4cc9ab32b320fbd3d52d in term 1.
I20260812 01:19:03.829124 28226 raft_consensus.cc:2781] T d036f772869f43cf9880eb8a2230faf4 P ff2989b8d8bd4cc9ab32b320fbd3d52d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:19:03.829282 28226 raft_consensus.cc:686] T d036f772869f43cf9880eb8a2230faf4 P ff2989b8d8bd4cc9ab32b320fbd3d52d [term 1 LEADER]: Becoming Leader. State: Replica: ff2989b8d8bd4cc9ab32b320fbd3d52d, State: Running, Role: LEADER
I20260812 01:19:03.829356 28226 consensus_queue.cc:237] T d036f772869f43cf9880eb8a2230faf4 P ff2989b8d8bd4cc9ab32b320fbd3d52d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff2989b8d8bd4cc9ab32b320fbd3d52d" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 45255 } } peers { permanent_uuid: "5969fd3075dd4de192f2fc1e163a42b0" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 41963 } } peers { permanent_uuid: "1da32ab00b8748aeb0b9879201a73b6d" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34529 } }
I20260812 01:19:03.830099 27354 catalog_manager.cc:5495] T d036f772869f43cf9880eb8a2230faf4 P ff2989b8d8bd4cc9ab32b320fbd3d52d reported cstate change: term changed from 0 to 1, leader changed from <none> to ff2989b8d8bd4cc9ab32b320fbd3d52d (127.24.0.129). New cstate: current_term: 1 leader_uuid: "ff2989b8d8bd4cc9ab32b320fbd3d52d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff2989b8d8bd4cc9ab32b320fbd3d52d" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 45255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5969fd3075dd4de192f2fc1e163a42b0" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 41963 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1da32ab00b8748aeb0b9879201a73b6d" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34529 } health_report { overall_health: UNKNOWN } } }
I20260812 01:19:03.857138 28206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:03.853924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35970 (local address 127.24.0.129:45255)
0812 01:19:03.853966 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:03.853968 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:03.853979 (+ 11us) server_negotiation.cc:407] Connection header received
0812 01:19:03.854048 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:03.854049 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:03.854071 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:03.854152 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:19:03.854238 (+ 86us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:03.855022 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:03.856110 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:03.856414 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:03.856471 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:03.856647 (+ 176us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:19:03.856780 (+ 133us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:19:03.856840 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:03.857036 (+ 196us) server_negotiation.cc:299] Negotiation successful
0812 01:19:03.857066 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 01:19:03.862478 28211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:03.859953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.129:51575 (local address 127.24.0.130:41963)
0812 01:19:03.860000 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:03.860002 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:03.860214 (+ 212us) server_negotiation.cc:407] Connection header received
0812 01:19:03.860247 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:03.860247 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:03.860258 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:03.860325 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:19:03.860428 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:03.861024 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:03.861759 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:03.862010 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:03.862079 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:03.862246 (+ 167us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:19:03.862281 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:19:03.862334 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:03.862386 (+ 52us) server_negotiation.cc:299] Negotiation successful
0812 01:19:03.862405 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 01:19:03.862501 28292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:03.859879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.130:41963 (local address 127.24.0.129:51575)
0812 01:19:03.860117 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:03.860129 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:03.860192 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:03.860331 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:03.860331 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:03.860332 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 01:19:03.860388 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:03.860389 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:03.861101 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:03.861102 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:03.861669 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:03.861670 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:03.862087 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:03.862087 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:03.862134 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:03.862147 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 01:19:03.862345 (+ 198us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 01:19:03.862346 (+ 1us) client_negotiation.cc:769] Sending connection context
0812 01:19:03.862381 (+ 35us) client_negotiation.cc:240] Negotiation successful
0812 01:19:03.862418 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":89,"threads_started":1}
I20260812 01:19:03.864964 27861 raft_consensus.cc:1261] T d036f772869f43cf9880eb8a2230faf4 P 5969fd3075dd4de192f2fc1e163a42b0 [term 1 FOLLOWER]: Refusing update from remote peer ff2989b8d8bd4cc9ab32b320fbd3d52d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:19:03.865190 28100 raft_consensus.cc:1261] T d036f772869f43cf9880eb8a2230faf4 P 1da32ab00b8748aeb0b9879201a73b6d [term 1 FOLLOWER]: Refusing update from remote peer ff2989b8d8bd4cc9ab32b320fbd3d52d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:19:03.865494 28226 consensus_queue.cc:1035] T d036f772869f43cf9880eb8a2230faf4 P ff2989b8d8bd4cc9ab32b320fbd3d52d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1da32ab00b8748aeb0b9879201a73b6d" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:19:03.865669 28284 consensus_queue.cc:1035] T d036f772869f43cf9880eb8a2230faf4 P ff2989b8d8bd4cc9ab32b320fbd3d52d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5969fd3075dd4de192f2fc1e163a42b0" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 41963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:19:03.873857 28206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:03.871401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.130:45427 (local address 127.24.0.129:45255)
0812 01:19:03.871438 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:03.871440 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:03.871451 (+ 11us) server_negotiation.cc:407] Connection header received
0812 01:19:03.871473 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:03.871474 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:03.871485 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:03.871548 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:19:03.871675 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:03.872357 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:03.873095 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:03.873348 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:03.873416 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:03.873593 (+ 177us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:19:03.873619 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:19:03.873690 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:03.873765 (+ 75us) server_negotiation.cc:299] Negotiation successful
0812 01:19:03.873785 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 01:19:03.873867 27953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:03.871316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.129:45255 (local address 127.24.0.130:45427)
0812 01:19:03.871363 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:03.871375 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:03.871417 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:03.871554 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:03.871555 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:03.871556 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 01:19:03.871607 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:03.871609 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:03.872441 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:03.872442 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:03.873003 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:03.873004 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:03.873443 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:03.873444 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:03.873510 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:03.873515 (+ 5us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 01:19:03.873707 (+ 192us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 01:19:03.873710 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 01:19:03.873760 (+ 50us) client_negotiation.cc:240] Negotiation successful
0812 01:19:03.873803 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23}
W20260812 01:19:03.874333 28253 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260812 01:19:05.395308 27354 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34760:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:19:05.395422 27354 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34760:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:19:05.396412 27354 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d3cf79f36f094ec0aee5f37e3cf9fe7c: Sending DeleteTablet for 3 replicas of tablet d036f772869f43cf9880eb8a2230faf4
I20260812 01:19:05.396721 27824 tablet_service.cc:1514] Processing DeleteTablet for tablet d036f772869f43cf9880eb8a2230faf4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:19:05 UTC) from {username='kudu', principal='kudu/127.24.0.190@KRBTEST.COM'} at 127.0.0.1:49626
I20260812 01:19:05.396750 27580 tablet_service.cc:1514] Processing DeleteTablet for tablet d036f772869f43cf9880eb8a2230faf4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:19:05 UTC) from {username='kudu', principal='kudu/127.24.0.190@KRBTEST.COM'} at 127.0.0.1:35958
I20260812 01:19:05.396927 28059 tablet_service.cc:1514] Processing DeleteTablet for tablet d036f772869f43cf9880eb8a2230faf4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:19:05 UTC) from {username='kudu', principal='kudu/127.24.0.190@KRBTEST.COM'} at 127.0.0.1:39502
I20260812 01:19:05.396957 28338 tablet_replica.cc:330] T d036f772869f43cf9880eb8a2230faf4 P 5969fd3075dd4de192f2fc1e163a42b0: stopping tablet replica
I20260812 01:19:05.397032 28338 raft_consensus.cc:2229] T d036f772869f43cf9880eb8a2230faf4 P 5969fd3075dd4de192f2fc1e163a42b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 01:19:05.397089 28338 raft_consensus.cc:2258] T d036f772869f43cf9880eb8a2230faf4 P 5969fd3075dd4de192f2fc1e163a42b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:19:05.397351 28338 ts_tablet_manager.cc:1905] T d036f772869f43cf9880eb8a2230faf4 P 5969fd3075dd4de192f2fc1e163a42b0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:19:05.399179 28342 tablet_replica.cc:330] T d036f772869f43cf9880eb8a2230faf4 P ff2989b8d8bd4cc9ab32b320fbd3d52d: stopping tablet replica
I20260812 01:19:05.399339 28343 tablet_replica.cc:330] T d036f772869f43cf9880eb8a2230faf4 P 1da32ab00b8748aeb0b9879201a73b6d: stopping tablet replica
I20260812 01:19:05.399387 28338 ts_tablet_manager.cc:1918] T d036f772869f43cf9880eb8a2230faf4 P 5969fd3075dd4de192f2fc1e163a42b0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 01:19:05.399451 28338 log.cc:1196] T d036f772869f43cf9880eb8a2230faf4 P 5969fd3075dd4de192f2fc1e163a42b0: Deleting WAL directory at /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/minicluster-data/ts-1/wal/wals/d036f772869f43cf9880eb8a2230faf4
I20260812 01:19:05.399577 28342 raft_consensus.cc:2229] T d036f772869f43cf9880eb8a2230faf4 P ff2989b8d8bd4cc9ab32b320fbd3d52d [term 1 LEADER]: Raft consensus shutting down.
I20260812 01:19:05.399712 28342 raft_consensus.cc:2258] T d036f772869f43cf9880eb8a2230faf4 P ff2989b8d8bd4cc9ab32b320fbd3d52d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:19:05.399719 28338 ts_tablet_manager.cc:1939] T d036f772869f43cf9880eb8a2230faf4 P 5969fd3075dd4de192f2fc1e163a42b0: Deleting consensus metadata
I20260812 01:19:05.399813 28343 raft_consensus.cc:2229] T d036f772869f43cf9880eb8a2230faf4 P 1da32ab00b8748aeb0b9879201a73b6d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 01:19:05.399876 28343 raft_consensus.cc:2258] T d036f772869f43cf9880eb8a2230faf4 P 1da32ab00b8748aeb0b9879201a73b6d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:19:05.400172 28343 ts_tablet_manager.cc:1905] T d036f772869f43cf9880eb8a2230faf4 P 1da32ab00b8748aeb0b9879201a73b6d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:19:05.400220 27329 catalog_manager.cc:4841] TS 5969fd3075dd4de192f2fc1e163a42b0 (127.24.0.130:41963): tablet d036f772869f43cf9880eb8a2230faf4 (table test-table [id=54b3071f313f4678a3b3bbec44922a69]) successfully deleted
I20260812 01:19:05.401414 28343 ts_tablet_manager.cc:1918] T d036f772869f43cf9880eb8a2230faf4 P 1da32ab00b8748aeb0b9879201a73b6d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 01:19:05.401469 28343 log.cc:1196] T d036f772869f43cf9880eb8a2230faf4 P 1da32ab00b8748aeb0b9879201a73b6d: Deleting WAL directory at /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/minicluster-data/ts-2/wal/wals/d036f772869f43cf9880eb8a2230faf4
I20260812 01:19:05.401666 28343 ts_tablet_manager.cc:1939] T d036f772869f43cf9880eb8a2230faf4 P 1da32ab00b8748aeb0b9879201a73b6d: Deleting consensus metadata
I20260812 01:19:05.401966 28342 ts_tablet_manager.cc:1905] T d036f772869f43cf9880eb8a2230faf4 P ff2989b8d8bd4cc9ab32b320fbd3d52d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:19:05.404141 28342 ts_tablet_manager.cc:1918] T d036f772869f43cf9880eb8a2230faf4 P ff2989b8d8bd4cc9ab32b320fbd3d52d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 01:19:05.404212 28342 log.cc:1196] T d036f772869f43cf9880eb8a2230faf4 P ff2989b8d8bd4cc9ab32b320fbd3d52d: Deleting WAL directory at /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/minicluster-data/ts-0/wal/wals/d036f772869f43cf9880eb8a2230faf4
I20260812 01:19:05.404332 27329 catalog_manager.cc:4841] TS 1da32ab00b8748aeb0b9879201a73b6d (127.24.0.131:34529): tablet d036f772869f43cf9880eb8a2230faf4 (table test-table [id=54b3071f313f4678a3b3bbec44922a69]) successfully deleted
I20260812 01:19:05.404523 28342 ts_tablet_manager.cc:1939] T d036f772869f43cf9880eb8a2230faf4 P ff2989b8d8bd4cc9ab32b320fbd3d52d: Deleting consensus metadata
I20260812 01:19:05.404874 27332 catalog_manager.cc:4841] TS ff2989b8d8bd4cc9ab32b320fbd3d52d (127.24.0.129:45255): tablet d036f772869f43cf9880eb8a2230faf4 (table test-table [id=54b3071f313f4678a3b3bbec44922a69]) successfully deleted
Aug 12 01:19:05 dist-test-slave-t3q3 krb5kdc[27285](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497545, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 01:19:05 dist-test-slave-t3q3 krb5kdc[27285](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497545, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
I20260812 01:19:05.425693 28362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:05.418470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34782 (local address 127.24.0.190:43905)
0812 01:19:05.418636 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:05.418639 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:05.418649 (+ 10us) server_negotiation.cc:407] Connection header received
0812 01:19:05.418723 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:05.418724 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:05.418738 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:05.418822 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:05.419451 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:05.420026 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:05.420909 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:05.421107 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:05.421162 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:05.423359 (+ 2197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:05.423360 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:05.423360 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:05.423367 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:05.424694 (+ 1327us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:05.425009 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:05.425010 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:05.425010 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:05.425067 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:05.425260 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:05.425260 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:05.425261 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:05.425365 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:05.425441 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:05.425554 (+ 113us) server_negotiation.cc:299] Negotiation successful
0812 01:19:05.425620 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":93,"threads_started":1}
I20260812 01:19:05.427059 27354 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34782:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 01:19:05.427268 27354 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 01:19:05.428721 27580 tablet_service.cc:1467] Processing CreateTablet for tablet db03478a94444520902453703ec13345 (DEFAULT_TABLE table=test-table [id=3b3103b166c04c0ab99ff9b96f9e5fbf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:19:05.428725 27824 tablet_service.cc:1467] Processing CreateTablet for tablet db03478a94444520902453703ec13345 (DEFAULT_TABLE table=test-table [id=3b3103b166c04c0ab99ff9b96f9e5fbf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:19:05.428881 27580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db03478a94444520902453703ec13345. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:19:05.428890 27824 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db03478a94444520902453703ec13345. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:19:05.429966 28059 tablet_service.cc:1467] Processing CreateTablet for tablet db03478a94444520902453703ec13345 (DEFAULT_TABLE table=test-table [id=3b3103b166c04c0ab99ff9b96f9e5fbf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:19:05.430023 28365 tablet_bootstrap.cc:492] T db03478a94444520902453703ec13345 P 5969fd3075dd4de192f2fc1e163a42b0: Bootstrap starting.
I20260812 01:19:05.430099 28059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db03478a94444520902453703ec13345. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:19:05.430487 28365 tablet_bootstrap.cc:654] T db03478a94444520902453703ec13345 P 5969fd3075dd4de192f2fc1e163a42b0: Neither blocks nor log segments found. Creating new log.
I20260812 01:19:05.431106 28365 tablet_bootstrap.cc:492] T db03478a94444520902453703ec13345 P 5969fd3075dd4de192f2fc1e163a42b0: No bootstrap required, opened a new log
I20260812 01:19:05.431147 28365 ts_tablet_manager.cc:1397] T db03478a94444520902453703ec13345 P 5969fd3075dd4de192f2fc1e163a42b0: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 01:19:05.431269 28365 raft_consensus.cc:348] T db03478a94444520902453703ec13345 P 5969fd3075dd4de192f2fc1e163a42b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff2989b8d8bd4cc9ab32b320fbd3d52d" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 45255 } } peers { permanent_uuid: "5969fd3075dd4de192f2fc1e163a42b0" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 41963 } } peers { permanent_uuid: "1da32ab00b8748aeb0b9879201a73b6d" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34529 } }
I20260812 01:19:05.431351 28365 raft_consensus.cc:374] T db03478a94444520902453703ec13345 P 5969fd3075dd4de192f2fc1e163a42b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:19:05.431372 28365 raft_consensus.cc:729] T db03478a94444520902453703ec13345 P 5969fd3075dd4de192f2fc1e163a42b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5969fd3075dd4de192f2fc1e163a42b0, State: Initialized, Role: FOLLOWER
I20260812 01:19:05.431421 28365 consensus_queue.cc:260] T db03478a94444520902453703ec13345 P 5969fd3075dd4de192f2fc1e163a42b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff2989b8d8bd4cc9ab32b320fbd3d52d" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 45255 } } peers { permanent_uuid: "5969fd3075dd4de192f2fc1e163a42b0" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 41963 } } peers { permanent_uuid: "1da32ab00b8748aeb0b9879201a73b6d" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34529 } }
I20260812 01:19:05.431367 28368 tablet_bootstrap.cc:492] T db03478a94444520902453703ec13345 P 1da32ab00b8748aeb0b9879201a73b6d: Bootstrap starting.
I20260812 01:19:05.431567 28365 ts_tablet_manager.cc:1428] T db03478a94444520902453703ec13345 P 5969fd3075dd4de192f2fc1e163a42b0: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 01:19:05.431886 28368 tablet_bootstrap.cc:654] T db03478a94444520902453703ec13345 P 1da32ab00b8748aeb0b9879201a73b6d: Neither blocks nor log segments found. Creating new log.
I20260812 01:19:05.431958 28363 tablet_bootstrap.cc:492] T db03478a94444520902453703ec13345 P ff2989b8d8bd4cc9ab32b320fbd3d52d: Bootstrap starting.
I20260812 01:19:05.432381 28363 tablet_bootstrap.cc:654] T db03478a94444520902453703ec13345 P ff2989b8d8bd4cc9ab32b320fbd3d52d: Neither blocks nor log segments found. Creating new log.
I20260812 01:19:05.432562 28368 tablet_bootstrap.cc:492] T db03478a94444520902453703ec13345 P 1da32ab00b8748aeb0b9879201a73b6d: No bootstrap required, opened a new log
I20260812 01:19:05.432607 28368 ts_tablet_manager.cc:1397] T db03478a94444520902453703ec13345 P 1da32ab00b8748aeb0b9879201a73b6d: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 01:19:05.432754 28368 raft_consensus.cc:348] T db03478a94444520902453703ec13345 P 1da32ab00b8748aeb0b9879201a73b6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff2989b8d8bd4cc9ab32b320fbd3d52d" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 45255 } } peers { permanent_uuid: "5969fd3075dd4de192f2fc1e163a42b0" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 41963 } } peers { permanent_uuid: "1da32ab00b8748aeb0b9879201a73b6d" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34529 } }
I20260812 01:19:05.432845 28368 raft_consensus.cc:374] T db03478a94444520902453703ec13345 P 1da32ab00b8748aeb0b9879201a73b6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:19:05.432873 28368 raft_consensus.cc:729] T db03478a94444520902453703ec13345 P 1da32ab00b8748aeb0b9879201a73b6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1da32ab00b8748aeb0b9879201a73b6d, State: Initialized, Role: FOLLOWER
I20260812 01:19:05.432919 28368 consensus_queue.cc:260] T db03478a94444520902453703ec13345 P 1da32ab00b8748aeb0b9879201a73b6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff2989b8d8bd4cc9ab32b320fbd3d52d" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 45255 } } peers { permanent_uuid: "5969fd3075dd4de192f2fc1e163a42b0" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 41963 } } peers { permanent_uuid: "1da32ab00b8748aeb0b9879201a73b6d" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34529 } }
I20260812 01:19:05.433090 28368 ts_tablet_manager.cc:1428] T db03478a94444520902453703ec13345 P 1da32ab00b8748aeb0b9879201a73b6d: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 01:19:05.433120 28363 tablet_bootstrap.cc:492] T db03478a94444520902453703ec13345 P ff2989b8d8bd4cc9ab32b320fbd3d52d: No bootstrap required, opened a new log
I20260812 01:19:05.433171 28363 ts_tablet_manager.cc:1397] T db03478a94444520902453703ec13345 P ff2989b8d8bd4cc9ab32b320fbd3d52d: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 01:19:05.433296 28363 raft_consensus.cc:348] T db03478a94444520902453703ec13345 P ff2989b8d8bd4cc9ab32b320fbd3d52d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff2989b8d8bd4cc9ab32b320fbd3d52d" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 45255 } } peers { permanent_uuid: "5969fd3075dd4de192f2fc1e163a42b0" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 41963 } } peers { permanent_uuid: "1da32ab00b8748aeb0b9879201a73b6d" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34529 } }
I20260812 01:19:05.433370 28363 raft_consensus.cc:374] T db03478a94444520902453703ec13345 P ff2989b8d8bd4cc9ab32b320fbd3d52d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:19:05.433394 28363 raft_consensus.cc:729] T db03478a94444520902453703ec13345 P ff2989b8d8bd4cc9ab32b320fbd3d52d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff2989b8d8bd4cc9ab32b320fbd3d52d, State: Initialized, Role: FOLLOWER
I20260812 01:19:05.433460 28363 consensus_queue.cc:260] T db03478a94444520902453703ec13345 P ff2989b8d8bd4cc9ab32b320fbd3d52d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff2989b8d8bd4cc9ab32b320fbd3d52d" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 45255 } } peers { permanent_uuid: "5969fd3075dd4de192f2fc1e163a42b0" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 41963 } } peers { permanent_uuid: "1da32ab00b8748aeb0b9879201a73b6d" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34529 } }
I20260812 01:19:05.433563 28363 ts_tablet_manager.cc:1428] T db03478a94444520902453703ec13345 P ff2989b8d8bd4cc9ab32b320fbd3d52d: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 01:19:05.725899 28373 raft_consensus.cc:482] T db03478a94444520902453703ec13345 P 1da32ab00b8748aeb0b9879201a73b6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 01:19:05.726028 28373 raft_consensus.cc:504] T db03478a94444520902453703ec13345 P 1da32ab00b8748aeb0b9879201a73b6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff2989b8d8bd4cc9ab32b320fbd3d52d" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 45255 } } peers { permanent_uuid: "5969fd3075dd4de192f2fc1e163a42b0" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 41963 } } peers { permanent_uuid: "1da32ab00b8748aeb0b9879201a73b6d" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34529 } }
I20260812 01:19:05.726812 28373 leader_election.cc:288] T db03478a94444520902453703ec13345 P 1da32ab00b8748aeb0b9879201a73b6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ff2989b8d8bd4cc9ab32b320fbd3d52d (127.24.0.129:45255), 5969fd3075dd4de192f2fc1e163a42b0 (127.24.0.130:41963)
I20260812 01:19:05.730993 28380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:05.726727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.129:45255 (local address 127.24.0.131:53481)
0812 01:19:05.727080 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:05.727090 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:05.727147 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:05.727326 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:05.727327 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:05.727328 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:19:05.727382 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:05.727384 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:05.728355 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:05.728356 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:05.729737 (+ 1381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:05.729740 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:05.730628 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:05.730629 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:05.730755 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:05.730760 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 01:19:05.730830 (+ 70us) client_negotiation.cc:240] Negotiation successful
0812 01:19:05.730891 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"mutex_wait_us":70,"thread_start_us":113,"threads_started":1}
I20260812 01:19:05.731078 28376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:05.726723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.131:37317 (local address 127.24.0.130:41963)
0812 01:19:05.727046 (+ 323us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:05.727049 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:05.727062 (+ 13us) server_negotiation.cc:407] Connection header received
0812 01:19:05.727127 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:05.727127 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:05.727140 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:05.727213 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:19:05.727413 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:05.728254 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:05.729844 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:05.730551 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:05.730634 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:05.730644 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:05.730930 (+ 286us) server_negotiation.cc:299] Negotiation successful
0812 01:19:05.730985 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":253,"thread_start_us":108,"threads_started":1}
I20260812 01:19:05.731257 28381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:05.726929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.131:53481 (local address 127.24.0.129:45255)
0812 01:19:05.727135 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:05.727137 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:05.727148 (+ 11us) server_negotiation.cc:407] Connection header received
0812 01:19:05.727244 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:05.727245 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:05.727257 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:05.727321 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:19:05.727465 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:05.728253 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:05.729845 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:05.730489 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:05.730576 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:05.730589 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:05.731109 (+ 520us) server_negotiation.cc:299] Negotiation successful
0812 01:19:05.731166 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":83,"threads_started":1}
I20260812 01:19:05.731276 28374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:05.726648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.130:41963 (local address 127.24.0.131:37317)
0812 01:19:05.726974 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:05.726987 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:05.727047 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:05.727248 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:05.727249 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:05.727252 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:19:05.727334 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:05.727336 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:05.728355 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:05.728356 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:05.729737 (+ 1381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:05.729740 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:05.730682 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:05.730683 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:05.730770 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:05.730775 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 01:19:05.730831 (+ 56us) client_negotiation.cc:240] Negotiation successful
0812 01:19:05.730894 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"mutex_wait_us":76,"thread_start_us":124,"threads_started":1}
I20260812 01:19:05.731426 27581 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db03478a94444520902453703ec13345" candidate_uuid: "1da32ab00b8748aeb0b9879201a73b6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff2989b8d8bd4cc9ab32b320fbd3d52d" is_pre_election: true
I20260812 01:19:05.731626 27581 raft_consensus.cc:2454] T db03478a94444520902453703ec13345 P ff2989b8d8bd4cc9ab32b320fbd3d52d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1da32ab00b8748aeb0b9879201a73b6d in term 0.
I20260812 01:19:05.731621 27861 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db03478a94444520902453703ec13345" candidate_uuid: "1da32ab00b8748aeb0b9879201a73b6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5969fd3075dd4de192f2fc1e163a42b0" is_pre_election: true
I20260812 01:19:05.731724 27861 raft_consensus.cc:2454] T db03478a94444520902453703ec13345 P 5969fd3075dd4de192f2fc1e163a42b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1da32ab00b8748aeb0b9879201a73b6d in term 0.
I20260812 01:19:05.731877 27984 leader_election.cc:302] T db03478a94444520902453703ec13345 P 1da32ab00b8748aeb0b9879201a73b6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1da32ab00b8748aeb0b9879201a73b6d, ff2989b8d8bd4cc9ab32b320fbd3d52d; no voters:
I20260812 01:19:05.732018 28373 raft_consensus.cc:2781] T db03478a94444520902453703ec13345 P 1da32ab00b8748aeb0b9879201a73b6d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 01:19:05.732049 28373 raft_consensus.cc:482] T db03478a94444520902453703ec13345 P 1da32ab00b8748aeb0b9879201a73b6d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 01:19:05.732112 28373 raft_consensus.cc:3037] T db03478a94444520902453703ec13345 P 1da32ab00b8748aeb0b9879201a73b6d [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:19:05.733134 28373 raft_consensus.cc:504] T db03478a94444520902453703ec13345 P 1da32ab00b8748aeb0b9879201a73b6d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff2989b8d8bd4cc9ab32b320fbd3d52d" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 45255 } } peers { permanent_uuid: "5969fd3075dd4de192f2fc1e163a42b0" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 41963 } } peers { permanent_uuid: "1da32ab00b8748aeb0b9879201a73b6d" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34529 } }
I20260812 01:19:05.733312 28373 leader_election.cc:288] T db03478a94444520902453703ec13345 P 1da32ab00b8748aeb0b9879201a73b6d [CANDIDATE]: Term 1 election: Requested vote from peers ff2989b8d8bd4cc9ab32b320fbd3d52d (127.24.0.129:45255), 5969fd3075dd4de192f2fc1e163a42b0 (127.24.0.130:41963)
I20260812 01:19:05.733472 27581 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db03478a94444520902453703ec13345" candidate_uuid: "1da32ab00b8748aeb0b9879201a73b6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff2989b8d8bd4cc9ab32b320fbd3d52d"
I20260812 01:19:05.733579 27581 raft_consensus.cc:3037] T db03478a94444520902453703ec13345 P ff2989b8d8bd4cc9ab32b320fbd3d52d [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:19:05.733592 27861 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db03478a94444520902453703ec13345" candidate_uuid: "1da32ab00b8748aeb0b9879201a73b6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5969fd3075dd4de192f2fc1e163a42b0"
I20260812 01:19:05.733660 27861 raft_consensus.cc:3037] T db03478a94444520902453703ec13345 P 5969fd3075dd4de192f2fc1e163a42b0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:19:05.734393 27861 raft_consensus.cc:2454] T db03478a94444520902453703ec13345 P 5969fd3075dd4de192f2fc1e163a42b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1da32ab00b8748aeb0b9879201a73b6d in term 1.
I20260812 01:19:05.734393 27581 raft_consensus.cc:2454] T db03478a94444520902453703ec13345 P ff2989b8d8bd4cc9ab32b320fbd3d52d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1da32ab00b8748aeb0b9879201a73b6d in term 1.
I20260812 01:19:05.734553 27984 leader_election.cc:302] T db03478a94444520902453703ec13345 P 1da32ab00b8748aeb0b9879201a73b6d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1da32ab00b8748aeb0b9879201a73b6d, ff2989b8d8bd4cc9ab32b320fbd3d52d; no voters:
I20260812 01:19:05.734654 28373 raft_consensus.cc:2781] T db03478a94444520902453703ec13345 P 1da32ab00b8748aeb0b9879201a73b6d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:19:05.735720 28373 raft_consensus.cc:686] T db03478a94444520902453703ec13345 P 1da32ab00b8748aeb0b9879201a73b6d [term 1 LEADER]: Becoming Leader. State: Replica: 1da32ab00b8748aeb0b9879201a73b6d, State: Running, Role: LEADER
I20260812 01:19:05.735808 28373 consensus_queue.cc:237] T db03478a94444520902453703ec13345 P 1da32ab00b8748aeb0b9879201a73b6d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff2989b8d8bd4cc9ab32b320fbd3d52d" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 45255 } } peers { permanent_uuid: "5969fd3075dd4de192f2fc1e163a42b0" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 41963 } } peers { permanent_uuid: "1da32ab00b8748aeb0b9879201a73b6d" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34529 } }
I20260812 01:19:05.736581 27354 catalog_manager.cc:5495] T db03478a94444520902453703ec13345 P 1da32ab00b8748aeb0b9879201a73b6d reported cstate change: term changed from 0 to 1, leader changed from <none> to 1da32ab00b8748aeb0b9879201a73b6d (127.24.0.131). New cstate: current_term: 1 leader_uuid: "1da32ab00b8748aeb0b9879201a73b6d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff2989b8d8bd4cc9ab32b320fbd3d52d" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 45255 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5969fd3075dd4de192f2fc1e163a42b0" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 41963 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1da32ab00b8748aeb0b9879201a73b6d" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34529 } health_report { overall_health: HEALTHY } } }
I20260812 01:19:05.790377 28391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:05.787660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39510 (local address 127.24.0.131:34529)
0812 01:19:05.787846 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:05.787848 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:05.787855 (+ 7us) server_negotiation.cc:407] Connection header received
0812 01:19:05.787892 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:05.787893 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:05.787913 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:05.787985 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:19:05.788184 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:05.788692 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:05.789556 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:05.789767 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:05.789842 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:05.790010 (+ 168us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:19:05.790125 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:19:05.790183 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:05.790231 (+ 48us) server_negotiation.cc:299] Negotiation successful
0812 01:19:05.790250 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":110,"threads_started":1}
I20260812 01:19:05.795509 28376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:05.793133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.131:50679 (local address 127.24.0.130:41963)
0812 01:19:05.793185 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:05.793187 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:05.793196 (+ 9us) server_negotiation.cc:407] Connection header received
0812 01:19:05.793262 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:05.793263 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:05.793274 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:05.793344 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:19:05.793495 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:05.793976 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:05.794785 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:05.794981 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:05.795063 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:05.795254 (+ 191us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:19:05.795290 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:19:05.795353 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:05.795409 (+ 56us) server_negotiation.cc:299] Negotiation successful
0812 01:19:05.795434 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 01:19:05.795547 28397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:05.792930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.130:41963 (local address 127.24.0.131:50679)
0812 01:19:05.793130 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:05.793141 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:05.793197 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:05.793348 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:05.793349 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:05.793352 (+ 3us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 01:19:05.793413 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:05.793414 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:05.794067 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:05.794068 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:05.794690 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:05.794692 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:05.795075 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:05.795076 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:05.795148 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:05.795167 (+ 19us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 01:19:05.795355 (+ 188us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 01:19:05.795358 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 01:19:05.795407 (+ 49us) client_negotiation.cc:240] Negotiation successful
0812 01:19:05.795451 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":126,"threads_started":1}
I20260812 01:19:05.797847 27581 raft_consensus.cc:1261] T db03478a94444520902453703ec13345 P ff2989b8d8bd4cc9ab32b320fbd3d52d [term 1 FOLLOWER]: Refusing update from remote peer 1da32ab00b8748aeb0b9879201a73b6d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:19:05.797847 27861 raft_consensus.cc:1261] T db03478a94444520902453703ec13345 P 5969fd3075dd4de192f2fc1e163a42b0 [term 1 FOLLOWER]: Refusing update from remote peer 1da32ab00b8748aeb0b9879201a73b6d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:19:05.798164 28382 consensus_queue.cc:1035] T db03478a94444520902453703ec13345 P 1da32ab00b8748aeb0b9879201a73b6d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5969fd3075dd4de192f2fc1e163a42b0" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 41963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:19:05.798318 28373 consensus_queue.cc:1035] T db03478a94444520902453703ec13345 P 1da32ab00b8748aeb0b9879201a73b6d [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff2989b8d8bd4cc9ab32b320fbd3d52d" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 45255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:19:05.805945 28391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:05.803343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.130:58089 (local address 127.24.0.131:34529)
0812 01:19:05.803409 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:05.803411 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:05.803516 (+ 105us) server_negotiation.cc:407] Connection header received
0812 01:19:05.803568 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:05.803569 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:05.803579 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:05.803636 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:19:05.803767 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:05.804363 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:05.805186 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:05.805445 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:05.805525 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:05.805676 (+ 151us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:19:05.805717 (+ 41us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:19:05.805786 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:05.805849 (+ 63us) server_negotiation.cc:299] Negotiation successful
0812 01:19:05.805870 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 01:19:05.805967 28403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:05.803308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.131:34529 (local address 127.24.0.130:58089)
0812 01:19:05.803431 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:05.803442 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:05.803506 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:05.803640 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:05.803641 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:05.803643 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 01:19:05.803696 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:05.803698 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:05.804431 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:05.804432 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:05.805081 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:05.805083 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:05.805523 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:05.805524 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:05.805585 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:05.805589 (+ 4us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 01:19:05.805788 (+ 199us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 01:19:05.805791 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 01:19:05.805844 (+ 53us) client_negotiation.cc:240] Negotiation successful
0812 01:19:05.805890 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":79,"threads_started":1}
I20260812 01:19:05.812827 27353 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34782:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:19:05.812952 27353 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34782:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:19:05.813683 27353 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d3cf79f36f094ec0aee5f37e3cf9fe7c: Sending DeleteTablet for 3 replicas of tablet db03478a94444520902453703ec13345
I20260812 01:19:05.813977 27580 tablet_service.cc:1514] Processing DeleteTablet for tablet db03478a94444520902453703ec13345 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:19:05 UTC) from {username='kudu', principal='kudu/127.24.0.190@KRBTEST.COM'} at 127.0.0.1:35958
I20260812 01:19:05.814090 28059 tablet_service.cc:1514] Processing DeleteTablet for tablet db03478a94444520902453703ec13345 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:19:05 UTC) from {username='kudu', principal='kudu/127.24.0.190@KRBTEST.COM'} at 127.0.0.1:39502
I20260812 01:19:05.814200 28342 tablet_replica.cc:330] T db03478a94444520902453703ec13345 P ff2989b8d8bd4cc9ab32b320fbd3d52d: stopping tablet replica
I20260812 01:19:05.814198 28343 tablet_replica.cc:330] T db03478a94444520902453703ec13345 P 1da32ab00b8748aeb0b9879201a73b6d: stopping tablet replica
I20260812 01:19:05.814249 28343 raft_consensus.cc:2229] T db03478a94444520902453703ec13345 P 1da32ab00b8748aeb0b9879201a73b6d [term 1 LEADER]: Raft consensus shutting down.
I20260812 01:19:05.814251 28342 raft_consensus.cc:2229] T db03478a94444520902453703ec13345 P ff2989b8d8bd4cc9ab32b320fbd3d52d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 01:19:05.814288 28342 raft_consensus.cc:2258] T db03478a94444520902453703ec13345 P ff2989b8d8bd4cc9ab32b320fbd3d52d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:19:05.814342 28343 raft_consensus.cc:2258] T db03478a94444520902453703ec13345 P 1da32ab00b8748aeb0b9879201a73b6d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:19:05.814424 27824 tablet_service.cc:1514] Processing DeleteTablet for tablet db03478a94444520902453703ec13345 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:19:05 UTC) from {username='kudu', principal='kudu/127.24.0.190@KRBTEST.COM'} at 127.0.0.1:49626
I20260812 01:19:05.814522 28338 tablet_replica.cc:330] T db03478a94444520902453703ec13345 P 5969fd3075dd4de192f2fc1e163a42b0: stopping tablet replica
I20260812 01:19:05.814576 28338 raft_consensus.cc:2229] T db03478a94444520902453703ec13345 P 5969fd3075dd4de192f2fc1e163a42b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 01:19:05.814616 28338 raft_consensus.cc:2258] T db03478a94444520902453703ec13345 P 5969fd3075dd4de192f2fc1e163a42b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:19:05.814682 28343 ts_tablet_manager.cc:1905] T db03478a94444520902453703ec13345 P 1da32ab00b8748aeb0b9879201a73b6d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:19:05.815068 28342 ts_tablet_manager.cc:1905] T db03478a94444520902453703ec13345 P ff2989b8d8bd4cc9ab32b320fbd3d52d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:19:05.816496 28343 ts_tablet_manager.cc:1918] T db03478a94444520902453703ec13345 P 1da32ab00b8748aeb0b9879201a73b6d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 01:19:05.816573 28343 log.cc:1196] T db03478a94444520902453703ec13345 P 1da32ab00b8748aeb0b9879201a73b6d: Deleting WAL directory at /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/minicluster-data/ts-2/wal/wals/db03478a94444520902453703ec13345
I20260812 01:19:05.816784 28342 ts_tablet_manager.cc:1918] T db03478a94444520902453703ec13345 P ff2989b8d8bd4cc9ab32b320fbd3d52d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 01:19:05.816833 28343 ts_tablet_manager.cc:1939] T db03478a94444520902453703ec13345 P 1da32ab00b8748aeb0b9879201a73b6d: Deleting consensus metadata
I20260812 01:19:05.816852 28342 log.cc:1196] T db03478a94444520902453703ec13345 P ff2989b8d8bd4cc9ab32b320fbd3d52d: Deleting WAL directory at /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/minicluster-data/ts-0/wal/wals/db03478a94444520902453703ec13345
I20260812 01:19:05.817056 28342 ts_tablet_manager.cc:1939] T db03478a94444520902453703ec13345 P ff2989b8d8bd4cc9ab32b320fbd3d52d: Deleting consensus metadata
I20260812 01:19:05.817310 27353 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34782:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260812 01:19:05.817382 27332 catalog_manager.cc:4841] TS ff2989b8d8bd4cc9ab32b320fbd3d52d (127.24.0.129:45255): tablet db03478a94444520902453703ec13345 (table test-table [id=3b3103b166c04c0ab99ff9b96f9e5fbf]) successfully deleted
W20260812 01:19:05.817485 27353 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 01:19:05.817481 28338 ts_tablet_manager.cc:1905] T db03478a94444520902453703ec13345 P 5969fd3075dd4de192f2fc1e163a42b0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:19:05.818912 27824 tablet_service.cc:1467] Processing CreateTablet for tablet 3ab5154bcca7489a9a71399d1ea6ce65 (DEFAULT_TABLE table=test-table [id=22db0e4d4ec24a8d9512ccf0e0090d61]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:19:05.818974 27580 tablet_service.cc:1467] Processing CreateTablet for tablet 3ab5154bcca7489a9a71399d1ea6ce65 (DEFAULT_TABLE table=test-table [id=22db0e4d4ec24a8d9512ccf0e0090d61]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:19:05.819053 27824 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ab5154bcca7489a9a71399d1ea6ce65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:19:05.819094 27580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ab5154bcca7489a9a71399d1ea6ce65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:19:05.819193 28338 ts_tablet_manager.cc:1918] T db03478a94444520902453703ec13345 P 5969fd3075dd4de192f2fc1e163a42b0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 01:19:05.819247 28338 log.cc:1196] T db03478a94444520902453703ec13345 P 5969fd3075dd4de192f2fc1e163a42b0: Deleting WAL directory at /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/minicluster-data/ts-1/wal/wals/db03478a94444520902453703ec13345
I20260812 01:19:05.819458 28338 ts_tablet_manager.cc:1939] T db03478a94444520902453703ec13345 P 5969fd3075dd4de192f2fc1e163a42b0: Deleting consensus metadata
I20260812 01:19:05.819882 28059 tablet_service.cc:1467] Processing CreateTablet for tablet 3ab5154bcca7489a9a71399d1ea6ce65 (DEFAULT_TABLE table=test-table [id=22db0e4d4ec24a8d9512ccf0e0090d61]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:19:05.820006 28059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ab5154bcca7489a9a71399d1ea6ce65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:19:05.820478 28363 tablet_bootstrap.cc:492] T 3ab5154bcca7489a9a71399d1ea6ce65 P ff2989b8d8bd4cc9ab32b320fbd3d52d: Bootstrap starting.
I20260812 01:19:05.820484 28365 tablet_bootstrap.cc:492] T 3ab5154bcca7489a9a71399d1ea6ce65 P 5969fd3075dd4de192f2fc1e163a42b0: Bootstrap starting.
I20260812 01:19:05.820766 28365 tablet_bootstrap.cc:654] T 3ab5154bcca7489a9a71399d1ea6ce65 P 5969fd3075dd4de192f2fc1e163a42b0: Neither blocks nor log segments found. Creating new log.
I20260812 01:19:05.820832 28363 tablet_bootstrap.cc:654] T 3ab5154bcca7489a9a71399d1ea6ce65 P ff2989b8d8bd4cc9ab32b320fbd3d52d: Neither blocks nor log segments found. Creating new log.
I20260812 01:19:05.821271 27329 catalog_manager.cc:4841] TS 5969fd3075dd4de192f2fc1e163a42b0 (127.24.0.130:41963): tablet db03478a94444520902453703ec13345 (table test-table [id=3b3103b166c04c0ab99ff9b96f9e5fbf]) successfully deleted
I20260812 01:19:05.821346 28368 tablet_bootstrap.cc:492] T 3ab5154bcca7489a9a71399d1ea6ce65 P 1da32ab00b8748aeb0b9879201a73b6d: Bootstrap starting.
I20260812 01:19:05.821375 27329 catalog_manager.cc:4841] TS 1da32ab00b8748aeb0b9879201a73b6d (127.24.0.131:34529): tablet db03478a94444520902453703ec13345 (table test-table [id=3b3103b166c04c0ab99ff9b96f9e5fbf]) successfully deleted
I20260812 01:19:05.821709 28363 tablet_bootstrap.cc:492] T 3ab5154bcca7489a9a71399d1ea6ce65 P ff2989b8d8bd4cc9ab32b320fbd3d52d: No bootstrap required, opened a new log
I20260812 01:19:05.821758 28363 ts_tablet_manager.cc:1397] T 3ab5154bcca7489a9a71399d1ea6ce65 P ff2989b8d8bd4cc9ab32b320fbd3d52d: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 01:19:05.821760 28368 tablet_bootstrap.cc:654] T 3ab5154bcca7489a9a71399d1ea6ce65 P 1da32ab00b8748aeb0b9879201a73b6d: Neither blocks nor log segments found. Creating new log.
I20260812 01:19:05.821858 28365 tablet_bootstrap.cc:492] T 3ab5154bcca7489a9a71399d1ea6ce65 P 5969fd3075dd4de192f2fc1e163a42b0: No bootstrap required, opened a new log
I20260812 01:19:05.821898 28365 ts_tablet_manager.cc:1397] T 3ab5154bcca7489a9a71399d1ea6ce65 P 5969fd3075dd4de192f2fc1e163a42b0: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 01:19:05.821888 28363 raft_consensus.cc:348] T 3ab5154bcca7489a9a71399d1ea6ce65 P ff2989b8d8bd4cc9ab32b320fbd3d52d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff2989b8d8bd4cc9ab32b320fbd3d52d" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 45255 } } peers { permanent_uuid: "5969fd3075dd4de192f2fc1e163a42b0" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 41963 } } peers { permanent_uuid: "1da32ab00b8748aeb0b9879201a73b6d" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34529 } }
I20260812 01:19:05.821966 28363 raft_consensus.cc:374] T 3ab5154bcca7489a9a71399d1ea6ce65 P ff2989b8d8bd4cc9ab32b320fbd3d52d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:19:05.821987 28363 raft_consensus.cc:729] T 3ab5154bcca7489a9a71399d1ea6ce65 P ff2989b8d8bd4cc9ab32b320fbd3d52d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff2989b8d8bd4cc9ab32b320fbd3d52d, State: Initialized, Role: FOLLOWER
I20260812 01:19:05.822034 28363 consensus_queue.cc:260] T 3ab5154bcca7489a9a71399d1ea6ce65 P ff2989b8d8bd4cc9ab32b320fbd3d52d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff2989b8d8bd4cc9ab32b320fbd3d52d" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 45255 } } peers { permanent_uuid: "5969fd3075dd4de192f2fc1e163a42b0" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 41963 } } peers { permanent_uuid: "1da32ab00b8748aeb0b9879201a73b6d" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34529 } }
I20260812 01:19:05.822036 28365 raft_consensus.cc:348] T 3ab5154bcca7489a9a71399d1ea6ce65 P 5969fd3075dd4de192f2fc1e163a42b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff2989b8d8bd4cc9ab32b320fbd3d52d" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 45255 } } peers { permanent_uuid: "5969fd3075dd4de192f2fc1e163a42b0" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 41963 } } peers { permanent_uuid: "1da32ab00b8748aeb0b9879201a73b6d" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34529 } }
I20260812 01:19:05.822104 28365 raft_consensus.cc:374] T 3ab5154bcca7489a9a71399d1ea6ce65 P 5969fd3075dd4de192f2fc1e163a42b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:19:05.822126 28365 raft_consensus.cc:729] T 3ab5154bcca7489a9a71399d1ea6ce65 P 5969fd3075dd4de192f2fc1e163a42b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5969fd3075dd4de192f2fc1e163a42b0, State: Initialized, Role: FOLLOWER
I20260812 01:19:05.822127 28363 ts_tablet_manager.cc:1428] T 3ab5154bcca7489a9a71399d1ea6ce65 P ff2989b8d8bd4cc9ab32b320fbd3d52d: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 01:19:05.822170 28365 consensus_queue.cc:260] T 3ab5154bcca7489a9a71399d1ea6ce65 P 5969fd3075dd4de192f2fc1e163a42b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff2989b8d8bd4cc9ab32b320fbd3d52d" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 45255 } } peers { permanent_uuid: "5969fd3075dd4de192f2fc1e163a42b0" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 41963 } } peers { permanent_uuid: "1da32ab00b8748aeb0b9879201a73b6d" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34529 } }
I20260812 01:19:05.822279 28365 ts_tablet_manager.cc:1428] T 3ab5154bcca7489a9a71399d1ea6ce65 P 5969fd3075dd4de192f2fc1e163a42b0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:05.822949 28368 tablet_bootstrap.cc:492] T 3ab5154bcca7489a9a71399d1ea6ce65 P 1da32ab00b8748aeb0b9879201a73b6d: No bootstrap required, opened a new log
I20260812 01:19:05.823005 28368 ts_tablet_manager.cc:1397] T 3ab5154bcca7489a9a71399d1ea6ce65 P 1da32ab00b8748aeb0b9879201a73b6d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 01:19:05.823163 28368 raft_consensus.cc:348] T 3ab5154bcca7489a9a71399d1ea6ce65 P 1da32ab00b8748aeb0b9879201a73b6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff2989b8d8bd4cc9ab32b320fbd3d52d" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 45255 } } peers { permanent_uuid: "5969fd3075dd4de192f2fc1e163a42b0" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 41963 } } peers { permanent_uuid: "1da32ab00b8748aeb0b9879201a73b6d" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34529 } }
I20260812 01:19:05.823246 28368 raft_consensus.cc:374] T 3ab5154bcca7489a9a71399d1ea6ce65 P 1da32ab00b8748aeb0b9879201a73b6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:19:05.823268 28368 raft_consensus.cc:729] T 3ab5154bcca7489a9a71399d1ea6ce65 P 1da32ab00b8748aeb0b9879201a73b6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1da32ab00b8748aeb0b9879201a73b6d, State: Initialized, Role: FOLLOWER
I20260812 01:19:05.823309 28368 consensus_queue.cc:260] T 3ab5154bcca7489a9a71399d1ea6ce65 P 1da32ab00b8748aeb0b9879201a73b6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff2989b8d8bd4cc9ab32b320fbd3d52d" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 45255 } } peers { permanent_uuid: "5969fd3075dd4de192f2fc1e163a42b0" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 41963 } } peers { permanent_uuid: "1da32ab00b8748aeb0b9879201a73b6d" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34529 } }
I20260812 01:19:05.823417 28368 ts_tablet_manager.cc:1428] T 3ab5154bcca7489a9a71399d1ea6ce65 P 1da32ab00b8748aeb0b9879201a73b6d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:05.837774 28400 raft_consensus.cc:482] T 3ab5154bcca7489a9a71399d1ea6ce65 P 1da32ab00b8748aeb0b9879201a73b6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 01:19:05.837886 28400 raft_consensus.cc:504] T 3ab5154bcca7489a9a71399d1ea6ce65 P 1da32ab00b8748aeb0b9879201a73b6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff2989b8d8bd4cc9ab32b320fbd3d52d" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 45255 } } peers { permanent_uuid: "5969fd3075dd4de192f2fc1e163a42b0" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 41963 } } peers { permanent_uuid: "1da32ab00b8748aeb0b9879201a73b6d" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34529 } }
I20260812 01:19:05.838109 28400 leader_election.cc:288] T 3ab5154bcca7489a9a71399d1ea6ce65 P 1da32ab00b8748aeb0b9879201a73b6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ff2989b8d8bd4cc9ab32b320fbd3d52d (127.24.0.129:45255), 5969fd3075dd4de192f2fc1e163a42b0 (127.24.0.130:41963)
I20260812 01:19:05.838304 27861 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ab5154bcca7489a9a71399d1ea6ce65" candidate_uuid: "1da32ab00b8748aeb0b9879201a73b6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5969fd3075dd4de192f2fc1e163a42b0" is_pre_election: true
I20260812 01:19:05.838325 27581 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ab5154bcca7489a9a71399d1ea6ce65" candidate_uuid: "1da32ab00b8748aeb0b9879201a73b6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff2989b8d8bd4cc9ab32b320fbd3d52d" is_pre_election: true
I20260812 01:19:05.838411 27861 raft_consensus.cc:2454] T 3ab5154bcca7489a9a71399d1ea6ce65 P 5969fd3075dd4de192f2fc1e163a42b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1da32ab00b8748aeb0b9879201a73b6d in term 0.
I20260812 01:19:05.838429 27581 raft_consensus.cc:2454] T 3ab5154bcca7489a9a71399d1ea6ce65 P ff2989b8d8bd4cc9ab32b320fbd3d52d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1da32ab00b8748aeb0b9879201a73b6d in term 0.
I20260812 01:19:05.838601 27984 leader_election.cc:302] T 3ab5154bcca7489a9a71399d1ea6ce65 P 1da32ab00b8748aeb0b9879201a73b6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1da32ab00b8748aeb0b9879201a73b6d, ff2989b8d8bd4cc9ab32b320fbd3d52d; no voters:
I20260812 01:19:05.838734 28400 raft_consensus.cc:2781] T 3ab5154bcca7489a9a71399d1ea6ce65 P 1da32ab00b8748aeb0b9879201a73b6d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 01:19:05.838790 28400 raft_consensus.cc:482] T 3ab5154bcca7489a9a71399d1ea6ce65 P 1da32ab00b8748aeb0b9879201a73b6d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 01:19:05.838815 28400 raft_consensus.cc:3037] T 3ab5154bcca7489a9a71399d1ea6ce65 P 1da32ab00b8748aeb0b9879201a73b6d [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:19:05.839547 28400 raft_consensus.cc:504] T 3ab5154bcca7489a9a71399d1ea6ce65 P 1da32ab00b8748aeb0b9879201a73b6d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff2989b8d8bd4cc9ab32b320fbd3d52d" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 45255 } } peers { permanent_uuid: "5969fd3075dd4de192f2fc1e163a42b0" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 41963 } } peers { permanent_uuid: "1da32ab00b8748aeb0b9879201a73b6d" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34529 } }
I20260812 01:19:05.839849 28400 leader_election.cc:288] T 3ab5154bcca7489a9a71399d1ea6ce65 P 1da32ab00b8748aeb0b9879201a73b6d [CANDIDATE]: Term 1 election: Requested vote from peers ff2989b8d8bd4cc9ab32b320fbd3d52d (127.24.0.129:45255), 5969fd3075dd4de192f2fc1e163a42b0 (127.24.0.130:41963)
I20260812 01:19:05.839835 27581 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ab5154bcca7489a9a71399d1ea6ce65" candidate_uuid: "1da32ab00b8748aeb0b9879201a73b6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff2989b8d8bd4cc9ab32b320fbd3d52d"
I20260812 01:19:05.839907 27581 raft_consensus.cc:3037] T 3ab5154bcca7489a9a71399d1ea6ce65 P ff2989b8d8bd4cc9ab32b320fbd3d52d [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:19:05.840214 27861 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ab5154bcca7489a9a71399d1ea6ce65" candidate_uuid: "1da32ab00b8748aeb0b9879201a73b6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5969fd3075dd4de192f2fc1e163a42b0"
I20260812 01:19:05.840293 27861 raft_consensus.cc:3037] T 3ab5154bcca7489a9a71399d1ea6ce65 P 5969fd3075dd4de192f2fc1e163a42b0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:19:05.840804 27581 raft_consensus.cc:2454] T 3ab5154bcca7489a9a71399d1ea6ce65 P ff2989b8d8bd4cc9ab32b320fbd3d52d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1da32ab00b8748aeb0b9879201a73b6d in term 1.
I20260812 01:19:05.840981 27984 leader_election.cc:302] T 3ab5154bcca7489a9a71399d1ea6ce65 P 1da32ab00b8748aeb0b9879201a73b6d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1da32ab00b8748aeb0b9879201a73b6d, ff2989b8d8bd4cc9ab32b320fbd3d52d; no voters:
I20260812 01:19:05.841082 28400 raft_consensus.cc:2781] T 3ab5154bcca7489a9a71399d1ea6ce65 P 1da32ab00b8748aeb0b9879201a73b6d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:19:05.841089 27861 raft_consensus.cc:2454] T 3ab5154bcca7489a9a71399d1ea6ce65 P 5969fd3075dd4de192f2fc1e163a42b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1da32ab00b8748aeb0b9879201a73b6d in term 1.
I20260812 01:19:05.841180 28400 raft_consensus.cc:686] T 3ab5154bcca7489a9a71399d1ea6ce65 P 1da32ab00b8748aeb0b9879201a73b6d [term 1 LEADER]: Becoming Leader. State: Replica: 1da32ab00b8748aeb0b9879201a73b6d, State: Running, Role: LEADER
I20260812 01:19:05.841230 28400 consensus_queue.cc:237] T 3ab5154bcca7489a9a71399d1ea6ce65 P 1da32ab00b8748aeb0b9879201a73b6d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff2989b8d8bd4cc9ab32b320fbd3d52d" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 45255 } } peers { permanent_uuid: "5969fd3075dd4de192f2fc1e163a42b0" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 41963 } } peers { permanent_uuid: "1da32ab00b8748aeb0b9879201a73b6d" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34529 } }
I20260812 01:19:05.842257 27353 catalog_manager.cc:5495] T 3ab5154bcca7489a9a71399d1ea6ce65 P 1da32ab00b8748aeb0b9879201a73b6d reported cstate change: term changed from 0 to 1, leader changed from <none> to 1da32ab00b8748aeb0b9879201a73b6d (127.24.0.131). New cstate: current_term: 1 leader_uuid: "1da32ab00b8748aeb0b9879201a73b6d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff2989b8d8bd4cc9ab32b320fbd3d52d" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 45255 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5969fd3075dd4de192f2fc1e163a42b0" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 41963 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1da32ab00b8748aeb0b9879201a73b6d" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34529 } health_report { overall_health: HEALTHY } } }
I20260812 01:19:05.850008 27861 raft_consensus.cc:1261] T 3ab5154bcca7489a9a71399d1ea6ce65 P 5969fd3075dd4de192f2fc1e163a42b0 [term 1 FOLLOWER]: Refusing update from remote peer 1da32ab00b8748aeb0b9879201a73b6d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:19:05.850021 27581 raft_consensus.cc:1261] T 3ab5154bcca7489a9a71399d1ea6ce65 P ff2989b8d8bd4cc9ab32b320fbd3d52d [term 1 FOLLOWER]: Refusing update from remote peer 1da32ab00b8748aeb0b9879201a73b6d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:19:05.850205 28400 consensus_queue.cc:1035] T 3ab5154bcca7489a9a71399d1ea6ce65 P 1da32ab00b8748aeb0b9879201a73b6d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5969fd3075dd4de192f2fc1e163a42b0" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 41963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:19:05.850319 28373 consensus_queue.cc:1035] T 3ab5154bcca7489a9a71399d1ea6ce65 P 1da32ab00b8748aeb0b9879201a73b6d [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff2989b8d8bd4cc9ab32b320fbd3d52d" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 45255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 01:19:05.855100 28304 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260812 01:19:07.373792 27353 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34782:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:19:07.373893 27353 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34782:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:19:07.374698 27353 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d3cf79f36f094ec0aee5f37e3cf9fe7c: Sending DeleteTablet for 3 replicas of tablet 3ab5154bcca7489a9a71399d1ea6ce65
I20260812 01:19:07.374935 27580 tablet_service.cc:1514] Processing DeleteTablet for tablet 3ab5154bcca7489a9a71399d1ea6ce65 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:19:07 UTC) from {username='kudu', principal='kudu/127.24.0.190@KRBTEST.COM'} at 127.0.0.1:35958
I20260812 01:19:07.374944 27824 tablet_service.cc:1514] Processing DeleteTablet for tablet 3ab5154bcca7489a9a71399d1ea6ce65 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:19:07 UTC) from {username='kudu', principal='kudu/127.24.0.190@KRBTEST.COM'} at 127.0.0.1:49626
I20260812 01:19:07.374970 28059 tablet_service.cc:1514] Processing DeleteTablet for tablet 3ab5154bcca7489a9a71399d1ea6ce65 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:19:07 UTC) from {username='kudu', principal='kudu/127.24.0.190@KRBTEST.COM'} at 127.0.0.1:39502
I20260812 01:19:07.375077 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 27469
I20260812 01:19:07.380939 28446 tablet_replica.cc:330] T 3ab5154bcca7489a9a71399d1ea6ce65 P 1da32ab00b8748aeb0b9879201a73b6d: stopping tablet replica
I20260812 01:19:07.381023 28446 raft_consensus.cc:2229] T 3ab5154bcca7489a9a71399d1ea6ce65 P 1da32ab00b8748aeb0b9879201a73b6d [term 1 LEADER]: Raft consensus shutting down.
I20260812 01:19:07.381100 28446 raft_consensus.cc:2258] T 3ab5154bcca7489a9a71399d1ea6ce65 P 1da32ab00b8748aeb0b9879201a73b6d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:19:07.381377 28446 ts_tablet_manager.cc:1905] T 3ab5154bcca7489a9a71399d1ea6ce65 P 1da32ab00b8748aeb0b9879201a73b6d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:19:07.381670 28444 tablet_replica.cc:330] T 3ab5154bcca7489a9a71399d1ea6ce65 P 5969fd3075dd4de192f2fc1e163a42b0: stopping tablet replica
I20260812 01:19:07.381739 28444 raft_consensus.cc:2229] T 3ab5154bcca7489a9a71399d1ea6ce65 P 5969fd3075dd4de192f2fc1e163a42b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 01:19:07.381778 28444 raft_consensus.cc:2258] T 3ab5154bcca7489a9a71399d1ea6ce65 P 5969fd3075dd4de192f2fc1e163a42b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:19:07.383020 28444 ts_tablet_manager.cc:1905] T 3ab5154bcca7489a9a71399d1ea6ce65 P 5969fd3075dd4de192f2fc1e163a42b0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:19:07.383185 28446 ts_tablet_manager.cc:1918] T 3ab5154bcca7489a9a71399d1ea6ce65 P 1da32ab00b8748aeb0b9879201a73b6d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 01:19:07.383251 28446 log.cc:1196] T 3ab5154bcca7489a9a71399d1ea6ce65 P 1da32ab00b8748aeb0b9879201a73b6d: Deleting WAL directory at /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/minicluster-data/ts-2/wal/wals/3ab5154bcca7489a9a71399d1ea6ce65
I20260812 01:19:07.383476 28446 ts_tablet_manager.cc:1939] T 3ab5154bcca7489a9a71399d1ea6ce65 P 1da32ab00b8748aeb0b9879201a73b6d: Deleting consensus metadata
I20260812 01:19:07.383846 27329 catalog_manager.cc:4841] TS 1da32ab00b8748aeb0b9879201a73b6d (127.24.0.131:34529): tablet 3ab5154bcca7489a9a71399d1ea6ce65 (table test-table [id=22db0e4d4ec24a8d9512ccf0e0090d61]) successfully deleted
I20260812 01:19:07.384740 28444 ts_tablet_manager.cc:1918] T 3ab5154bcca7489a9a71399d1ea6ce65 P 5969fd3075dd4de192f2fc1e163a42b0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 01:19:07.384795 28444 log.cc:1196] T 3ab5154bcca7489a9a71399d1ea6ce65 P 5969fd3075dd4de192f2fc1e163a42b0: Deleting WAL directory at /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786497534195564-24578-0/minicluster-data/ts-1/wal/wals/3ab5154bcca7489a9a71399d1ea6ce65
I20260812 01:19:07.385012 28444 ts_tablet_manager.cc:1939] T 3ab5154bcca7489a9a71399d1ea6ce65 P 5969fd3075dd4de192f2fc1e163a42b0: Deleting consensus metadata
I20260812 01:19:07.385339 27329 catalog_manager.cc:4841] TS 5969fd3075dd4de192f2fc1e163a42b0 (127.24.0.130:41963): tablet 3ab5154bcca7489a9a71399d1ea6ce65 (table test-table [id=22db0e4d4ec24a8d9512ccf0e0090d61]) successfully deleted
W20260812 01:19:07.389847 27332 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.0.129:45255 (error 108)
I20260812 01:19:07.390048 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 27702
I20260812 01:19:07.399019 28448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:07.397785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.129:45255 (local address 127.0.0.1:35984)
0812 01:19:07.398928 (+ 1143us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:07.398979 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.129:45255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1121,"thread_start_us":835,"threads_started":1}
W20260812 01:19:07.399156 27332 catalog_manager.cc:4584] TS ff2989b8d8bd4cc9ab32b320fbd3d52d (127.24.0.129:45255): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3ab5154bcca7489a9a71399d1ea6ce65: Network error: Client connection negotiation failed: client connection to 127.24.0.129:45255: connect: Connection refused (error 111)
I20260812 01:19:07.405301 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 27957
I20260812 01:19:07.420346 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 27304
2026-08-12T01:19:07Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6112 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786497534195564-24578-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 01:19:07 dist-test-slave-t3q3 krb5kdc[28454](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 01:19:07 dist-test-slave-t3q3 krb5kdc[28454](info): set up 2 sockets
Aug 12 01:19:07 dist-test-slave-t3q3 krb5kdc[28454](info): commencing operation
krb5kdc: starting...
W20260812 01:19:08.694339 24578 mini_kdc.cc:121] Time spent starting KDC: real 1.245s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786497534195564-24578-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786497534195564-24578-0/krb5kdc/test-user.keytab.
Aug 12 01:19:08 dist-test-slave-t3q3 krb5kdc[28454](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497548, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T01:19:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:19:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.0.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.0.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.0.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab.
Entry for principal kudu/127.24.0.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab.
I20260812 01:19:08.820866 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786497534195564-24578-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786497534195564-24578-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786497534195564-24578-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.190:41409
--webserver_interface=127.24.0.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab
--principal=kudu/127.24.0.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:33921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.0.190:41409
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:19:08.905304 28475 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:19:08.905544 28475 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:19:08.905586 28475 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:19:08.906708 28475 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:19:08.906746 28475 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:19:08.906761 28475 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:19:08.906773 28475 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:19:08.906785 28475 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:19:08.908198 28475 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:33921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786497534195564-24578-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786497534195564-24578-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.0.190:41409
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab
--principal=kudu/127.24.0.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.190:41409
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.0.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28475
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786497534195564-24578-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:19:08.908385 28475 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:19:08.908550 28475 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:19:08.913311 28480 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:08.913323 28481 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:08.913333 28484 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:19:08.913661 28475 server_base.cc:1034] running on GCE node
I20260812 01:19:08.913921 28475 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:19:08.914172 28475 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:19:08.915313 28475 hybrid_clock.cc:648] HybridClock initialized: now 1786497548915302 us; error 29 us; skew 500 ppm
Aug 12 01:19:08 dist-test-slave-t3q3 krb5kdc[28454](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497548, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:19:08.922022 28475 init.cc:377] Logged in from keytab as kudu/127.24.0.190@KRBTEST.COM (short username kudu)
I20260812 01:19:08.922466 28475 webserver.cc:460] Webserver started at http://127.24.0.190:41937/ using document root <none> and password file <none>
I20260812 01:19:08.922675 28475 fs_manager.cc:359] Metadata directory not provided
I20260812 01:19:08.922731 28475 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:19:08.922818 28475 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:19:08.923825 28475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786497534195564-24578-0/minicluster-data/master-0/data/instance:
uuid: "48ad17961299421fbcdf180b41ce1953"
format_stamp: "Formatted at 2026-08-12 01:19:08 on dist-test-slave-t3q3"
server_key: "16ce3a0fe5ea841fba57cdea83772eef"
server_key_iv: "f972f3603394957b634c5a379756a29a"
server_key_version: "encryptionkey@0"
I20260812 01:19:08.924197 28475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786497534195564-24578-0/minicluster-data/master-0/wal/instance:
uuid: "48ad17961299421fbcdf180b41ce1953"
format_stamp: "Formatted at 2026-08-12 01:19:08 on dist-test-slave-t3q3"
server_key: "16ce3a0fe5ea841fba57cdea83772eef"
server_key_iv: "f972f3603394957b634c5a379756a29a"
server_key_version: "encryptionkey@0"
I20260812 01:19:08.925467 28475 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:19:08.926174 28504 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:08.926371 28475 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 01:19:08.926451 28475 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786497534195564-24578-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786497534195564-24578-0/minicluster-data/master-0/wal
uuid: "48ad17961299421fbcdf180b41ce1953"
format_stamp: "Formatted at 2026-08-12 01:19:08 on dist-test-slave-t3q3"
server_key: "16ce3a0fe5ea841fba57cdea83772eef"
server_key_iv: "f972f3603394957b634c5a379756a29a"
server_key_version: "encryptionkey@0"
I20260812 01:19:08.926503 28475 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786497534195564-24578-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786497534195564-24578-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786497534195564-24578-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:19:08.943542 28475 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:19:08.945343 28475 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:19:08.945454 28475 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:19:08.949518 28475 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.190:41409
I20260812 01:19:08.949574 28588 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.190:41409 every 8 connection(s)
I20260812 01:19:08.949915 28475 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
I20260812 01:19:08.950767 28589 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:19:08.953051 28589 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 48ad17961299421fbcdf180b41ce1953: Bootstrap starting.
I20260812 01:19:08.953636 28589 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 48ad17961299421fbcdf180b41ce1953: Neither blocks nor log segments found. Creating new log.
I20260812 01:19:08.953943 28589 log.cc:824] T 00000000000000000000000000000000 P 48ad17961299421fbcdf180b41ce1953: Log is configured to *not* fsync() on all Append() calls
I20260812 01:19:08.954635 28589 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 48ad17961299421fbcdf180b41ce1953: No bootstrap required, opened a new log
I20260812 01:19:08.956328 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 28475
I20260812 01:19:08.956408 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786497534195564-24578-0/minicluster-data/master-0/wal/instance
I20260812 01:19:08.956322 28589 raft_consensus.cc:348] T 00000000000000000000000000000000 P 48ad17961299421fbcdf180b41ce1953 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48ad17961299421fbcdf180b41ce1953" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 41409 } }
I20260812 01:19:08.956524 28589 raft_consensus.cc:374] T 00000000000000000000000000000000 P 48ad17961299421fbcdf180b41ce1953 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:19:08.956593 28589 raft_consensus.cc:729] T 00000000000000000000000000000000 P 48ad17961299421fbcdf180b41ce1953 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48ad17961299421fbcdf180b41ce1953, State: Initialized, Role: FOLLOWER
I20260812 01:19:08.956616 24578 external_mini_cluster.cc:1428] Setting key 3ce41025cfc0ae35907de7c0a95d04c5
I20260812 01:19:08.956777 28589 consensus_queue.cc:260] T 00000000000000000000000000000000 P 48ad17961299421fbcdf180b41ce1953 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48ad17961299421fbcdf180b41ce1953" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 41409 } }
I20260812 01:19:08.956888 28589 raft_consensus.cc:388] T 00000000000000000000000000000000 P 48ad17961299421fbcdf180b41ce1953 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:19:08.956936 28589 raft_consensus.cc:482] T 00000000000000000000000000000000 P 48ad17961299421fbcdf180b41ce1953 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:19:08.956987 28589 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 48ad17961299421fbcdf180b41ce1953 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:19:08.958110 28589 raft_consensus.cc:504] T 00000000000000000000000000000000 P 48ad17961299421fbcdf180b41ce1953 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48ad17961299421fbcdf180b41ce1953" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 41409 } }
I20260812 01:19:08.958256 28589 leader_election.cc:302] T 00000000000000000000000000000000 P 48ad17961299421fbcdf180b41ce1953 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 48ad17961299421fbcdf180b41ce1953; no voters:
I20260812 01:19:08.958431 28589 leader_election.cc:288] T 00000000000000000000000000000000 P 48ad17961299421fbcdf180b41ce1953 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 01:19:08.958496 28596 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 48ad17961299421fbcdf180b41ce1953 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:19:08.958650 28596 raft_consensus.cc:686] T 00000000000000000000000000000000 P 48ad17961299421fbcdf180b41ce1953 [term 1 LEADER]: Becoming Leader. State: Replica: 48ad17961299421fbcdf180b41ce1953, State: Running, Role: LEADER
I20260812 01:19:08.958806 28589 sys_catalog.cc:564] T 00000000000000000000000000000000 P 48ad17961299421fbcdf180b41ce1953 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:19:08.958797 28596 consensus_queue.cc:237] T 00000000000000000000000000000000 P 48ad17961299421fbcdf180b41ce1953 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48ad17961299421fbcdf180b41ce1953" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 41409 } }
I20260812 01:19:08.959372 28602 sys_catalog.cc:455] T 00000000000000000000000000000000 P 48ad17961299421fbcdf180b41ce1953 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 48ad17961299421fbcdf180b41ce1953. Latest consensus state: current_term: 1 leader_uuid: "48ad17961299421fbcdf180b41ce1953" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48ad17961299421fbcdf180b41ce1953" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 41409 } } }
I20260812 01:19:08.959451 28602 sys_catalog.cc:458] T 00000000000000000000000000000000 P 48ad17961299421fbcdf180b41ce1953 [sys.catalog]: This master's current role is: LEADER
I20260812 01:19:08.959648 28598 sys_catalog.cc:455] T 00000000000000000000000000000000 P 48ad17961299421fbcdf180b41ce1953 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "48ad17961299421fbcdf180b41ce1953" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48ad17961299421fbcdf180b41ce1953" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 41409 } } }
I20260812 01:19:08.959745 28598 sys_catalog.cc:458] T 00000000000000000000000000000000 P 48ad17961299421fbcdf180b41ce1953 [sys.catalog]: This master's current role is: LEADER
I20260812 01:19:08.960192 28614 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:19:08.960738 28614 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 12 01:19:08 dist-test-slave-t3q3 krb5kdc[28454](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497548, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
I20260812 01:19:08.964530 28614 catalog_manager.cc:1347] Generated new cluster ID: baccbaff3bb1464d9ace7286b1bc5b5f
I20260812 01:19:08.964584 28614 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 01:19:08.968334 28595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:08.957144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57788 (local address 127.24.0.190:41409)
0812 01:19:08.957355 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:08.957368 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:08.957394 (+ 26us) server_negotiation.cc:407] Connection header received
0812 01:19:08.957497 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:08.957503 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:08.957543 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:08.957653 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:08.958537 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:08.959980 (+ 1443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:08.961220 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:08.961826 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:08.961966 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:08.964462 (+ 2496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:08.964466 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:08.964466 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:08.964479 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:08.966627 (+ 2148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:08.967036 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:08.967040 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:08.967040 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:08.967166 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:08.967378 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:08.967379 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:08.967379 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:08.967709 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:08.967878 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:08.967959 (+ 81us) server_negotiation.cc:299] Negotiation successful
0812 01:19:08.968129 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":127,"threads_started":1}
I20260812 01:19:08.974948 28614 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:19:08.975471 28614 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:19:08.981422 28614 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 48ad17961299421fbcdf180b41ce1953: Generated new TSK 0
I20260812 01:19:08.981603 28614 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.0.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.0.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.0.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.129.keytab.
Entry for principal kudu/127.24.0.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.129.keytab.
I20260812 01:19:09.010438 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786497534195564-24578-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786497534195564-24578-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786497534195564-24578-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786497534195564-24578-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.0.129:0
--local_ip_for_outbound_sockets=127.24.0.129
--webserver_interface=127.24.0.129
--webserver_port=0
--tserver_master_addrs=127.24.0.190:41409
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.129.keytab
--principal=kudu/127.24.0.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:33921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:19:09.094609 28640 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:19:09.094828 28640 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:19:09.094853 28640 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:19:09.096100 28640 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:19:09.096139 28640 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:19:09.096165 28640 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.0.129
I20260812 01:19:09.097617 28640 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:33921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786497534195564-24578-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786497534195564-24578-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.129.keytab
--principal=kudu/127.24.0.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.0.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786497534195564-24578-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.0.129
--webserver_port=0
--tserver_master_addrs=127.24.0.190:41409
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28640
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.0.129
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786497534195564-24578-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:19:09.097839 28640 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:19:09.098066 28640 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:19:09.103367 28649 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:09.103376 28647 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:19:09.103528 28640 server_base.cc:1034] running on GCE node
W20260812 01:19:09.103376 28646 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:19:09.103825 28640 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:19:09.104137 28640 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:19:09.105309 28640 hybrid_clock.cc:648] HybridClock initialized: now 1786497549105294 us; error 45 us; skew 500 ppm
Aug 12 01:19:09 dist-test-slave-t3q3 krb5kdc[28454](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497549, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:19:09.107841 28640 init.cc:377] Logged in from keytab as kudu/127.24.0.129@KRBTEST.COM (short username kudu)
I20260812 01:19:09.108407 28640 webserver.cc:460] Webserver started at http://127.24.0.129:42105/ using document root <none> and password file <none>
I20260812 01:19:09.108651 28640 fs_manager.cc:359] Metadata directory not provided
I20260812 01:19:09.108722 28640 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:19:09.108879 28640 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:19:09.109999 28640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786497534195564-24578-0/minicluster-data/ts-0/data/instance:
uuid: "789d6ca027ed4b97910ed8d33812ded0"
format_stamp: "Formatted at 2026-08-12 01:19:09 on dist-test-slave-t3q3"
server_key: "00739dfd2cc1d50a3949e39dd4955626"
server_key_iv: "4e4f7b3ad6ce72e3e94ed2422f22523e"
server_key_version: "encryptionkey@0"
I20260812 01:19:09.110381 28640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786497534195564-24578-0/minicluster-data/ts-0/wal/instance:
uuid: "789d6ca027ed4b97910ed8d33812ded0"
format_stamp: "Formatted at 2026-08-12 01:19:09 on dist-test-slave-t3q3"
server_key: "00739dfd2cc1d50a3949e39dd4955626"
server_key_iv: "4e4f7b3ad6ce72e3e94ed2422f22523e"
server_key_version: "encryptionkey@0"
I20260812 01:19:09.111743 28640 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 01:19:09.112560 28658 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:09.112740 28640 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:09.112849 28640 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786497534195564-24578-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786497534195564-24578-0/minicluster-data/ts-0/wal
uuid: "789d6ca027ed4b97910ed8d33812ded0"
format_stamp: "Formatted at 2026-08-12 01:19:09 on dist-test-slave-t3q3"
server_key: "00739dfd2cc1d50a3949e39dd4955626"
server_key_iv: "4e4f7b3ad6ce72e3e94ed2422f22523e"
server_key_version: "encryptionkey@0"
I20260812 01:19:09.112926 28640 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786497534195564-24578-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786497534195564-24578-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786497534195564-24578-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:19:09.129467 28640 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:19:09.131682 28640 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:19:09.131875 28640 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:19:09.132205 28640 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:19:09.132558 28640 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:19:09.132613 28640 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:09.132648 28640 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:19:09.132675 28640 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:09.138815 28640 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.129:43183
I20260812 01:19:09.138887 28845 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.129:43183 every 8 connection(s)
I20260812 01:19:09.139261 28640 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786497534195564-24578-0/minicluster-data/ts-0/data/info.pb
Aug 12 01:19:09 dist-test-slave-t3q3 krb5kdc[28454](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497549, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.129@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
I20260812 01:19:09.146585 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 28640
I20260812 01:19:09.146693 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786497534195564-24578-0/minicluster-data/ts-0/wal/instance
I20260812 01:19:09.146903 24578 external_mini_cluster.cc:1428] Setting key 2a59b7d706ebff201363c9b7febf7c0c
I20260812 01:19:09.149430 28595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:09.140125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.129:58437 (local address 127.24.0.190:41409)
0812 01:19:09.140242 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:09.140245 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:09.140431 (+ 186us) server_negotiation.cc:407] Connection header received
0812 01:19:09.140980 (+ 549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:09.140981 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:09.140999 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:09.141074 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:09.142100 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:09.142626 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:09.143645 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:09.143863 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:09.143955 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:09.146227 (+ 2272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:09.146229 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:09.146229 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:09.146237 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:09.147873 (+ 1636us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:09.148409 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:09.148410 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:09.148411 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:09.148475 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:09.148745 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:09.148746 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:09.148747 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:09.148868 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:09.148944 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:09.149232 (+ 288us) server_negotiation.cc:299] Negotiation successful
0812 01:19:09.149337 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260812 01:19:09.149614 28851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:09.140035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:41409 (local address 127.24.0.129:58437)
0812 01:19:09.140293 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:09.140331 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:09.140866 (+ 535us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:09.141152 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:09.141156 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:09.141758 (+ 602us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:09.141962 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:09.141967 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:09.142728 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:09.142729 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:09.143548 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:09.143549 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:09.143967 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:09.143969 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:09.144123 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:09.144614 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:09.144623 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:09.146086 (+ 1463us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:19:09.147983 (+ 1897us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:09.147991 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:09.147993 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:09.148318 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:09.148548 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:09.148549 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:09.148549 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:09.148665 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:09.148935 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:19:09.148942 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:19:09.149089 (+ 147us) client_negotiation.cc:769] Sending connection context
0812 01:19:09.149190 (+ 101us) client_negotiation.cc:240] Negotiation successful
0812 01:19:09.149360 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":106,"threads_started":1}
I20260812 01:19:09.150175 28848 heartbeater.cc:347] Connected to a master server at 127.24.0.190:41409
I20260812 01:19:09.150365 28848 heartbeater.cc:464] Registering TS with master...
I20260812 01:19:09.150578 28848 heartbeater.cc:511] Master 127.24.0.190:41409 requested a full tablet report, sending...
I20260812 01:19:09.151211 28534 ts_manager.cc:194] Registered new tserver with Master: 789d6ca027ed4b97910ed8d33812ded0 (127.24.0.129:43183)
I20260812 01:19:09.151793 28534 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.0.129@KRBTEST.COM'} at 127.24.0.129:58437
WARNING: no policy specified for kudu/127.24.0.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.0.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.0.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.130.keytab.
Entry for principal kudu/127.24.0.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.130.keytab.
I20260812 01:19:09.174223 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786497534195564-24578-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786497534195564-24578-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786497534195564-24578-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786497534195564-24578-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.0.130:0
--local_ip_for_outbound_sockets=127.24.0.130
--webserver_interface=127.24.0.130
--webserver_port=0
--tserver_master_addrs=127.24.0.190:41409
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.130.keytab
--principal=kudu/127.24.0.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:33921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:19:09.256966 28856 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:19:09.257159 28856 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:19:09.257184 28856 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:19:09.258268 28856 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:19:09.258301 28856 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:19:09.258325 28856 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.0.130
I20260812 01:19:09.259914 28856 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:33921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786497534195564-24578-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786497534195564-24578-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.130.keytab
--principal=kudu/127.24.0.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.0.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786497534195564-24578-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.0.130
--webserver_port=0
--tserver_master_addrs=127.24.0.190:41409
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28856
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.0.130
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786497534195564-24578-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:19:09.260167 28856 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:19:09.260357 28856 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 01:19:09.265278 28856 server_base.cc:1034] running on GCE node
W20260812 01:19:09.265213 28864 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:09.265213 28867 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:09.265236 28863 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:19:09.265630 28856 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:19:09.265875 28856 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:19:09.267016 28856 hybrid_clock.cc:648] HybridClock initialized: now 1786497549266999 us; error 38 us; skew 500 ppm
Aug 12 01:19:09 dist-test-slave-t3q3 krb5kdc[28454](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497549, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:19:09.269557 28856 init.cc:377] Logged in from keytab as kudu/127.24.0.130@KRBTEST.COM (short username kudu)
I20260812 01:19:09.270030 28856 webserver.cc:460] Webserver started at http://127.24.0.130:41825/ using document root <none> and password file <none>
I20260812 01:19:09.270237 28856 fs_manager.cc:359] Metadata directory not provided
I20260812 01:19:09.270278 28856 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:19:09.270368 28856 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:19:09.271347 28856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786497534195564-24578-0/minicluster-data/ts-1/data/instance:
uuid: "01061271140642a980f7d80049205af9"
format_stamp: "Formatted at 2026-08-12 01:19:09 on dist-test-slave-t3q3"
server_key: "dc182a6063ef13563666c6e747c46cec"
server_key_iv: "404d5c21656c78d9f0a96525badd6262"
server_key_version: "encryptionkey@0"
I20260812 01:19:09.271652 28856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786497534195564-24578-0/minicluster-data/ts-1/wal/instance:
uuid: "01061271140642a980f7d80049205af9"
format_stamp: "Formatted at 2026-08-12 01:19:09 on dist-test-slave-t3q3"
server_key: "dc182a6063ef13563666c6e747c46cec"
server_key_iv: "404d5c21656c78d9f0a96525badd6262"
server_key_version: "encryptionkey@0"
I20260812 01:19:09.273001 28856 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 01:19:09.273720 28881 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:09.273895 28856 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 01:19:09.273989 28856 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786497534195564-24578-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786497534195564-24578-0/minicluster-data/ts-1/wal
uuid: "01061271140642a980f7d80049205af9"
format_stamp: "Formatted at 2026-08-12 01:19:09 on dist-test-slave-t3q3"
server_key: "dc182a6063ef13563666c6e747c46cec"
server_key_iv: "404d5c21656c78d9f0a96525badd6262"
server_key_version: "encryptionkey@0"
I20260812 01:19:09.274071 28856 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786497534195564-24578-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786497534195564-24578-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786497534195564-24578-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:19:09.287968 28856 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:19:09.290449 28856 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:19:09.290618 28856 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:19:09.290876 28856 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:19:09.291195 28856 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:19:09.291252 28856 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:09.291297 28856 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:19:09.291322 28856 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:09.297430 28856 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.130:42111
I20260812 01:19:09.297478 29065 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.130:42111 every 8 connection(s)
I20260812 01:19:09.297845 28856 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786497534195564-24578-0/minicluster-data/ts-1/data/info.pb
I20260812 01:19:09.299634 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 28856
I20260812 01:19:09.299754 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786497534195564-24578-0/minicluster-data/ts-1/wal/instance
I20260812 01:19:09.299973 24578 external_mini_cluster.cc:1428] Setting key f632004a49c5397c1c4ceccd6dee46c6
Aug 12 01:19:09 dist-test-slave-t3q3 krb5kdc[28454](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497549, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.130@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
I20260812 01:19:09.308283 28595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:09.298606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.130:56205 (local address 127.24.0.190:41409)
0812 01:19:09.298674 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:09.298676 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:09.298877 (+ 201us) server_negotiation.cc:407] Connection header received
0812 01:19:09.299336 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:09.299337 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:09.299348 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:09.299418 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:09.300729 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:09.301219 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:09.302707 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:09.302937 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:09.303024 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:09.305169 (+ 2145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:09.305171 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:09.305171 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:09.305180 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:09.306867 (+ 1687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:09.307254 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:09.307255 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:09.307256 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:09.307328 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:09.307566 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:09.307567 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:09.307568 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:09.307725 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:09.307805 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:09.308041 (+ 236us) server_negotiation.cc:299] Negotiation successful
0812 01:19:09.308164 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 01:19:09.308416 29069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:09.298589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:41409 (local address 127.24.0.130:56205)
0812 01:19:09.298753 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:09.298788 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:09.299253 (+ 465us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:09.299493 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:09.299498 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:09.300368 (+ 870us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:09.300617 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:09.300623 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:09.301311 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:09.301312 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:09.302616 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:09.302619 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:09.303035 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:09.303036 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:09.303168 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:09.303675 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:09.303683 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:09.305072 (+ 1389us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:19:09.306950 (+ 1878us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:09.306955 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:09.306957 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:09.307167 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:09.307400 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:09.307401 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:09.307401 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:09.307495 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:09.307797 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:19:09.307801 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:19:09.307909 (+ 108us) client_negotiation.cc:769] Sending connection context
0812 01:19:09.307988 (+ 79us) client_negotiation.cc:240] Negotiation successful
0812 01:19:09.308163 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":93,"threads_started":1}
I20260812 01:19:09.308918 29066 heartbeater.cc:347] Connected to a master server at 127.24.0.190:41409
I20260812 01:19:09.309031 29066 heartbeater.cc:464] Registering TS with master...
I20260812 01:19:09.309221 29066 heartbeater.cc:511] Master 127.24.0.190:41409 requested a full tablet report, sending...
I20260812 01:19:09.309609 28534 ts_manager.cc:194] Registered new tserver with Master: 01061271140642a980f7d80049205af9 (127.24.0.130:42111)
WARNING: no policy specified for kudu/127.24.0.131@KRBTEST.COM; defaulting to no policy
I20260812 01:19:09.310110 28534 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.0.130@KRBTEST.COM'} at 127.24.0.130:56205
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.0.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.0.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.131.keytab.
Entry for principal kudu/127.24.0.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.131.keytab.
I20260812 01:19:09.326566 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786497534195564-24578-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786497534195564-24578-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786497534195564-24578-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786497534195564-24578-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.0.131:0
--local_ip_for_outbound_sockets=127.24.0.131
--webserver_interface=127.24.0.131
--webserver_port=0
--tserver_master_addrs=127.24.0.190:41409
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.131.keytab
--principal=kudu/127.24.0.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:33921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:19:09.410794 29077 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:19:09.411062 29077 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:19:09.411092 29077 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:19:09.412207 29077 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:19:09.412241 29077 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:19:09.412266 29077 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.0.131
I20260812 01:19:09.413631 29077 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:33921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786497534195564-24578-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786497534195564-24578-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.131.keytab
--principal=kudu/127.24.0.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.0.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786497534195564-24578-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.0.131
--webserver_port=0
--tserver_master_addrs=127.24.0.190:41409
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29077
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.0.131
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786497534195564-24578-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:19:09.413816 29077 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:19:09.413982 29077 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:19:09.419081 29086 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:19:09.419255 29077 server_base.cc:1034] running on GCE node
W20260812 01:19:09.419139 29089 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:09.419361 29087 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:19:09.419554 29077 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:19:09.419796 29077 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:19:09.420960 29077 hybrid_clock.cc:648] HybridClock initialized: now 1786497549420948 us; error 29 us; skew 500 ppm
Aug 12 01:19:09 dist-test-slave-t3q3 krb5kdc[28454](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497549, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:19:09.423328 29077 init.cc:377] Logged in from keytab as kudu/127.24.0.131@KRBTEST.COM (short username kudu)
I20260812 01:19:09.423800 29077 webserver.cc:460] Webserver started at http://127.24.0.131:45299/ using document root <none> and password file <none>
I20260812 01:19:09.424036 29077 fs_manager.cc:359] Metadata directory not provided
I20260812 01:19:09.424140 29077 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:19:09.424248 29077 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:19:09.425185 29077 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786497534195564-24578-0/minicluster-data/ts-2/data/instance:
uuid: "ec0059f5fa164314aa43303785399d41"
format_stamp: "Formatted at 2026-08-12 01:19:09 on dist-test-slave-t3q3"
server_key: "3c68aebb64dc2d8f5b65459fe46cb6be"
server_key_iv: "638e91d974ed790e54e8918cb8bb63c0"
server_key_version: "encryptionkey@0"
I20260812 01:19:09.425494 29077 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786497534195564-24578-0/minicluster-data/ts-2/wal/instance:
uuid: "ec0059f5fa164314aa43303785399d41"
format_stamp: "Formatted at 2026-08-12 01:19:09 on dist-test-slave-t3q3"
server_key: "3c68aebb64dc2d8f5b65459fe46cb6be"
server_key_iv: "638e91d974ed790e54e8918cb8bb63c0"
server_key_version: "encryptionkey@0"
I20260812 01:19:09.426713 29077 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 01:19:09.427384 29101 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:09.427569 29077 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 01:19:09.427641 29077 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786497534195564-24578-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786497534195564-24578-0/minicluster-data/ts-2/wal
uuid: "ec0059f5fa164314aa43303785399d41"
format_stamp: "Formatted at 2026-08-12 01:19:09 on dist-test-slave-t3q3"
server_key: "3c68aebb64dc2d8f5b65459fe46cb6be"
server_key_iv: "638e91d974ed790e54e8918cb8bb63c0"
server_key_version: "encryptionkey@0"
I20260812 01:19:09.427731 29077 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786497534195564-24578-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786497534195564-24578-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786497534195564-24578-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:19:09.438263 29077 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:19:09.440052 29077 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:19:09.440284 29077 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:19:09.440510 29077 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:19:09.440824 29077 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:19:09.440879 29077 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:09.440919 29077 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:19:09.440948 29077 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:09.446815 29077 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.131:42743
I20260812 01:19:09.446893 29292 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.131:42743 every 8 connection(s)
I20260812 01:19:09.447220 29077 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786497534195564-24578-0/minicluster-data/ts-2/data/info.pb
I20260812 01:19:09.451668 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 29077
I20260812 01:19:09.451782 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786497534195564-24578-0/minicluster-data/ts-2/wal/instance
I20260812 01:19:09.452306 24578 external_mini_cluster.cc:1428] Setting key 164284914ef607a5714f6fb5ce469c94
Aug 12 01:19:09 dist-test-slave-t3q3 krb5kdc[28454](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497549, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.131@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
I20260812 01:19:09.456561 28595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:09.447946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.131:54407 (local address 127.24.0.190:41409)
0812 01:19:09.448033 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:09.448035 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:09.448255 (+ 220us) server_negotiation.cc:407] Connection header received
0812 01:19:09.448722 (+ 467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:09.448724 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:09.448734 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:09.448787 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:09.449744 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:09.450258 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:09.451188 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:09.451366 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:09.451417 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:09.453929 (+ 2512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:09.453931 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:09.453931 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:09.453939 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:09.455137 (+ 1198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:09.455546 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:09.455547 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:09.455548 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:09.455608 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:09.455872 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:09.455873 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:09.455873 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:09.455990 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:09.456063 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:09.456387 (+ 324us) server_negotiation.cc:299] Negotiation successful
0812 01:19:09.456473 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 01:19:09.456739 29297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:09.447902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:41409 (local address 127.24.0.131:54407)
0812 01:19:09.448117 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:09.448173 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:09.448630 (+ 457us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:09.448841 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:09.448844 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:09.449453 (+ 609us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:09.449654 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:09.449660 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:09.450332 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:09.450333 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:09.451113 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:09.451115 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:09.451439 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:09.451439 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:09.451604 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:09.452225 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:09.452234 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:09.453833 (+ 1599us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:19:09.455230 (+ 1397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:09.455238 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:09.455240 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:09.455466 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:09.455687 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:09.455688 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:09.455688 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:09.455797 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:09.456083 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:19:09.456090 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:19:09.456241 (+ 151us) client_negotiation.cc:769] Sending connection context
0812 01:19:09.456347 (+ 106us) client_negotiation.cc:240] Negotiation successful
0812 01:19:09.456503 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":116,"threads_started":1}
I20260812 01:19:09.457233 29293 heartbeater.cc:347] Connected to a master server at 127.24.0.190:41409
I20260812 01:19:09.457340 29293 heartbeater.cc:464] Registering TS with master...
I20260812 01:19:09.457546 29293 heartbeater.cc:511] Master 127.24.0.190:41409 requested a full tablet report, sending...
I20260812 01:19:09.457921 28534 ts_manager.cc:194] Registered new tserver with Master: ec0059f5fa164314aa43303785399d41 (127.24.0.131:42743)
I20260812 01:19:09.458261 28534 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.0.131@KRBTEST.COM'} at 127.24.0.131:54407
I20260812 01:19:09.464190 24578 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260812 01:19:09.476751 28595 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:19:09.475687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35814 (local address 127.24.0.190:41409)
0812 01:19:09.475763 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:09.475766 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:09.475883 (+ 117us) server_negotiation.cc:407] Connection header received
0812 01:19:09.475972 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:09.475973 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:09.475984 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:09.476051 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:09.476667 (+ 616us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35814: BlockingRecv error: recv got EOF from 127.0.0.1:35814 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 01:19:09.477664 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 28640
I20260812 01:19:09.492630 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 28856
I20260812 01:19:09.507654 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 29077
I20260812 01:19:09.521409 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 28475
2026-08-12T01:19:09Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2100 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786497534195564-24578-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 01:19:09 dist-test-slave-t3q3 krb5kdc[29311](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 01:19:09 dist-test-slave-t3q3 krb5kdc[29311](info): set up 2 sockets
Aug 12 01:19:09 dist-test-slave-t3q3 krb5kdc[29311](info): commencing operation
krb5kdc: starting...
W20260812 01:19:10.804901 24578 mini_kdc.cc:121] Time spent starting KDC: real 1.255s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786497534195564-24578-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786497534195564-24578-0/krb5kdc/test-user.keytab.
Aug 12 01:19:10 dist-test-slave-t3q3 krb5kdc[29311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497550, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T01:19:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:19:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.0.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.0.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.0.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab.
Entry for principal kudu/127.24.0.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab.
I20260812 01:19:10.929008 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786497534195564-24578-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786497534195564-24578-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786497534195564-24578-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.190:45529
--webserver_interface=127.24.0.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab
--principal=kudu/127.24.0.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:37051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.0.190:45529
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:19:11.015302 29333 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:19:11.015517 29333 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:19:11.015542 29333 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:19:11.016824 29333 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:19:11.016866 29333 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:19:11.016881 29333 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:19:11.016918 29333 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:19:11.016947 29333 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:19:11.018471 29333 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:37051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786497534195564-24578-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786497534195564-24578-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.0.190:45529
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab
--principal=kudu/127.24.0.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.190:45529
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.0.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29333
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786497534195564-24578-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:19:11.018733 29333 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:19:11.018989 29333 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:19:11.024163 29345 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:11.024184 29342 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:11.024158 29341 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:19:11.024260 29333 server_base.cc:1034] running on GCE node
I20260812 01:19:11.024684 29333 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:19:11.025000 29333 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:19:11.026158 29333 hybrid_clock.cc:648] HybridClock initialized: now 1786497551026143 us; error 31 us; skew 500 ppm
Aug 12 01:19:11 dist-test-slave-t3q3 krb5kdc[29311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497551, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:19:11.028824 29333 init.cc:377] Logged in from keytab as kudu/127.24.0.190@KRBTEST.COM (short username kudu)
I20260812 01:19:11.029301 29333 webserver.cc:460] Webserver started at http://127.24.0.190:33551/ using document root <none> and password file <none>
I20260812 01:19:11.029582 29333 fs_manager.cc:359] Metadata directory not provided
I20260812 01:19:11.029654 29333 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:19:11.029812 29333 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:19:11.031010 29333 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786497534195564-24578-0/minicluster-data/master-0/data/instance:
uuid: "dbf4bec336304b51a90d9805290c0163"
format_stamp: "Formatted at 2026-08-12 01:19:11 on dist-test-slave-t3q3"
server_key: "d63fb58fa0449579d9570c2a9da2602e"
server_key_iv: "50c74039ecdb8171524dc58940412ad6"
server_key_version: "encryptionkey@0"
I20260812 01:19:11.031392 29333 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786497534195564-24578-0/minicluster-data/master-0/wal/instance:
uuid: "dbf4bec336304b51a90d9805290c0163"
format_stamp: "Formatted at 2026-08-12 01:19:11 on dist-test-slave-t3q3"
server_key: "d63fb58fa0449579d9570c2a9da2602e"
server_key_iv: "50c74039ecdb8171524dc58940412ad6"
server_key_version: "encryptionkey@0"
I20260812 01:19:11.032959 29333 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 01:19:11.033818 29363 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:11.034063 29333 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:19:11.034168 29333 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786497534195564-24578-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786497534195564-24578-0/minicluster-data/master-0/wal
uuid: "dbf4bec336304b51a90d9805290c0163"
format_stamp: "Formatted at 2026-08-12 01:19:11 on dist-test-slave-t3q3"
server_key: "d63fb58fa0449579d9570c2a9da2602e"
server_key_iv: "50c74039ecdb8171524dc58940412ad6"
server_key_version: "encryptionkey@0"
I20260812 01:19:11.034257 29333 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786497534195564-24578-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786497534195564-24578-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786497534195564-24578-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:19:11.041703 29333 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:19:11.043457 29333 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:19:11.043566 29333 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:19:11.047645 29333 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.190:45529
I20260812 01:19:11.047725 29459 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.190:45529 every 8 connection(s)
I20260812 01:19:11.048288 29333 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
I20260812 01:19:11.048981 29460 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:19:11.051226 29460 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dbf4bec336304b51a90d9805290c0163: Bootstrap starting.
I20260812 01:19:11.051822 29460 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dbf4bec336304b51a90d9805290c0163: Neither blocks nor log segments found. Creating new log.
I20260812 01:19:11.052239 29460 log.cc:824] T 00000000000000000000000000000000 P dbf4bec336304b51a90d9805290c0163: Log is configured to *not* fsync() on all Append() calls
I20260812 01:19:11.052955 29460 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dbf4bec336304b51a90d9805290c0163: No bootstrap required, opened a new log
I20260812 01:19:11.054273 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 29333
I20260812 01:19:11.054381 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786497534195564-24578-0/minicluster-data/master-0/wal/instance
I20260812 01:19:11.054548 24578 external_mini_cluster.cc:1428] Setting key fc159fa58a6ebf53f37d2600b7884a04
I20260812 01:19:11.054946 29460 raft_consensus.cc:348] T 00000000000000000000000000000000 P dbf4bec336304b51a90d9805290c0163 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbf4bec336304b51a90d9805290c0163" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 45529 } }
I20260812 01:19:11.055133 29460 raft_consensus.cc:374] T 00000000000000000000000000000000 P dbf4bec336304b51a90d9805290c0163 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:19:11.055218 29460 raft_consensus.cc:729] T 00000000000000000000000000000000 P dbf4bec336304b51a90d9805290c0163 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbf4bec336304b51a90d9805290c0163, State: Initialized, Role: FOLLOWER
I20260812 01:19:11.055377 29460 consensus_queue.cc:260] T 00000000000000000000000000000000 P dbf4bec336304b51a90d9805290c0163 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbf4bec336304b51a90d9805290c0163" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 45529 } }
I20260812 01:19:11.055456 29460 raft_consensus.cc:388] T 00000000000000000000000000000000 P dbf4bec336304b51a90d9805290c0163 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:19:11.055480 29460 raft_consensus.cc:482] T 00000000000000000000000000000000 P dbf4bec336304b51a90d9805290c0163 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:19:11.055547 29460 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dbf4bec336304b51a90d9805290c0163 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:19:11.056160 29460 raft_consensus.cc:504] T 00000000000000000000000000000000 P dbf4bec336304b51a90d9805290c0163 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbf4bec336304b51a90d9805290c0163" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 45529 } }
I20260812 01:19:11.056277 29460 leader_election.cc:302] T 00000000000000000000000000000000 P dbf4bec336304b51a90d9805290c0163 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dbf4bec336304b51a90d9805290c0163; no voters:
I20260812 01:19:11.056450 29460 leader_election.cc:288] T 00000000000000000000000000000000 P dbf4bec336304b51a90d9805290c0163 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 01:19:11.056502 29469 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dbf4bec336304b51a90d9805290c0163 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:19:11.056654 29469 raft_consensus.cc:686] T 00000000000000000000000000000000 P dbf4bec336304b51a90d9805290c0163 [term 1 LEADER]: Becoming Leader. State: Replica: dbf4bec336304b51a90d9805290c0163, State: Running, Role: LEADER
I20260812 01:19:11.056726 29460 sys_catalog.cc:564] T 00000000000000000000000000000000 P dbf4bec336304b51a90d9805290c0163 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:19:11.056777 29469 consensus_queue.cc:237] T 00000000000000000000000000000000 P dbf4bec336304b51a90d9805290c0163 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbf4bec336304b51a90d9805290c0163" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 45529 } }
I20260812 01:19:11.057247 29472 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbf4bec336304b51a90d9805290c0163 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dbf4bec336304b51a90d9805290c0163" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbf4bec336304b51a90d9805290c0163" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 45529 } } }
I20260812 01:19:11.057339 29472 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbf4bec336304b51a90d9805290c0163 [sys.catalog]: This master's current role is: LEADER
I20260812 01:19:11.057574 29476 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbf4bec336304b51a90d9805290c0163 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dbf4bec336304b51a90d9805290c0163. Latest consensus state: current_term: 1 leader_uuid: "dbf4bec336304b51a90d9805290c0163" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbf4bec336304b51a90d9805290c0163" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 45529 } } }
I20260812 01:19:11.057883 29476 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbf4bec336304b51a90d9805290c0163 [sys.catalog]: This master's current role is: LEADER
W20260812 01:19:11.058671 29492 catalog_manager.cc:1558] T 00000000000000000000000000000000 P dbf4bec336304b51a90d9805290c0163: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 01:19:11.058768 29492 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 01:19:11.058897 29482 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:19:11.059100 29482 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:19:11.060608 29482 catalog_manager.cc:1347] Generated new cluster ID: 9ba7370657dc44d18a2c3c469edf34b0
I20260812 01:19:11.060703 29482 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 01:19:11 dist-test-slave-t3q3 krb5kdc[29311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497550, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
I20260812 01:19:11.066681 29467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:11.055061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54482 (local address 127.24.0.190:45529)
0812 01:19:11.055748 (+ 687us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:11.055755 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:11.055780 (+ 25us) server_negotiation.cc:407] Connection header received
0812 01:19:11.055881 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:11.055886 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:11.055926 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:11.056034 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:11.056960 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:11.058481 (+ 1521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:11.059422 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:11.060031 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:11.060175 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:11.063057 (+ 2882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:11.063061 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:11.063061 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:11.063073 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:11.064999 (+ 1926us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:11.065460 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:11.065463 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:11.065463 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:11.065586 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:11.065807 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:11.065808 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:11.065809 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:11.066102 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:11.066260 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:11.066372 (+ 112us) server_negotiation.cc:299] Negotiation successful
0812 01:19:11.066498 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":630,"thread_start_us":584,"threads_started":1}
I20260812 01:19:11.067694 24578 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 01:19:11.076534 29482 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:19:11.077018 29482 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:19:11.085150 29482 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dbf4bec336304b51a90d9805290c0163: Generated new TSK 0
I20260812 01:19:11.085346 29482 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 01:19:11.085690 29467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:11.081113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54496 (local address 127.24.0.190:45529)
0812 01:19:11.081148 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:11.081149 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:11.081315 (+ 166us) server_negotiation.cc:407] Connection header received
0812 01:19:11.081350 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:11.081351 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:11.081363 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:11.081430 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:11.081809 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:11.082879 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:11.083896 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:11.084273 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:11.084339 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:11.085215 (+ 876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:11.085216 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:11.085216 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:19:11.085422 (+ 206us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:11.085439 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:19:11.085446 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:11.085508 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:11.085591 (+ 83us) server_negotiation.cc:299] Negotiation successful
0812 01:19:11.085615 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260812 01:19:11.088331 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 29333
2026-08-12T01:19:11Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1566 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786497534195564-24578-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 01:19:11 dist-test-slave-t3q3 krb5kdc[29509](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 01:19:11 dist-test-slave-t3q3 krb5kdc[29509](info): set up 2 sockets
Aug 12 01:19:11 dist-test-slave-t3q3 krb5kdc[29509](info): commencing operation
krb5kdc: starting...
W20260812 01:19:12.360702 24578 mini_kdc.cc:121] Time spent starting KDC: real 1.244s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786497534195564-24578-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786497534195564-24578-0/krb5kdc/test-user.keytab.
Aug 12 01:19:12 dist-test-slave-t3q3 krb5kdc[29509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497552, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T01:19:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:19:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.0.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.0.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.0.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab.
Entry for principal kudu/127.24.0.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab.
I20260812 01:19:12.500806 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786497534195564-24578-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786497534195564-24578-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786497534195564-24578-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.190:36597
--webserver_interface=127.24.0.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab
--principal=kudu/127.24.0.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:36551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.0.190:36597
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:19:12.583736 29532 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:19:12.583941 29532 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:19:12.583964 29532 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:19:12.585146 29532 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:19:12.585189 29532 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:19:12.585203 29532 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:19:12.585256 29532 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:19:12.585284 29532 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:19:12.586823 29532 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:36551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786497534195564-24578-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786497534195564-24578-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.0.190:36597
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab
--principal=kudu/127.24.0.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.190:36597
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.0.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29532
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786497534195564-24578-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:19:12.587108 29532 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:19:12.587397 29532 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:19:12.592243 29540 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:12.592221 29545 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:19:12.592372 29532 server_base.cc:1034] running on GCE node
W20260812 01:19:12.592226 29548 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:19:12.592780 29532 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:19:12.593096 29532 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:19:12.594241 29532 hybrid_clock.cc:648] HybridClock initialized: now 1786497552594233 us; error 37 us; skew 500 ppm
Aug 12 01:19:12 dist-test-slave-t3q3 krb5kdc[29509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497552, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:19:12.596866 29532 init.cc:377] Logged in from keytab as kudu/127.24.0.190@KRBTEST.COM (short username kudu)
I20260812 01:19:12.597321 29532 webserver.cc:460] Webserver started at http://127.24.0.190:41487/ using document root <none> and password file <none>
I20260812 01:19:12.597501 29532 fs_manager.cc:359] Metadata directory not provided
I20260812 01:19:12.597541 29532 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:19:12.597628 29532 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:19:12.598646 29532 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786497534195564-24578-0/minicluster-data/master-0/data/instance:
uuid: "506475378344442183192042916610d5"
format_stamp: "Formatted at 2026-08-12 01:19:12 on dist-test-slave-t3q3"
server_key: "8aaf2f9fa0eb7c2436c5a91d64e653d0"
server_key_iv: "405dbf971a4838fc943f4d12cf09ed25"
server_key_version: "encryptionkey@0"
I20260812 01:19:12.598958 29532 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786497534195564-24578-0/minicluster-data/master-0/wal/instance:
uuid: "506475378344442183192042916610d5"
format_stamp: "Formatted at 2026-08-12 01:19:12 on dist-test-slave-t3q3"
server_key: "8aaf2f9fa0eb7c2436c5a91d64e653d0"
server_key_iv: "405dbf971a4838fc943f4d12cf09ed25"
server_key_version: "encryptionkey@0"
I20260812 01:19:12.600324 29532 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 01:19:12.601068 29558 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:12.601255 29532 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:19:12.601366 29532 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786497534195564-24578-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786497534195564-24578-0/minicluster-data/master-0/wal
uuid: "506475378344442183192042916610d5"
format_stamp: "Formatted at 2026-08-12 01:19:12 on dist-test-slave-t3q3"
server_key: "8aaf2f9fa0eb7c2436c5a91d64e653d0"
server_key_iv: "405dbf971a4838fc943f4d12cf09ed25"
server_key_version: "encryptionkey@0"
I20260812 01:19:12.601444 29532 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786497534195564-24578-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786497534195564-24578-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786497534195564-24578-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:19:12.626648 29532 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:19:12.629067 29532 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:19:12.629246 29532 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:19:12.633510 29532 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.190:36597
I20260812 01:19:12.633553 29651 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.190:36597 every 8 connection(s)
I20260812 01:19:12.633932 29532 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
I20260812 01:19:12.634999 29653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:19:12.636291 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 29532
I20260812 01:19:12.636417 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786497534195564-24578-0/minicluster-data/master-0/wal/instance
I20260812 01:19:12.636600 24578 external_mini_cluster.cc:1428] Setting key a08505b58ac1560e1cef83374ecc79fa
I20260812 01:19:12.637820 29653 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 506475378344442183192042916610d5: Bootstrap starting.
I20260812 01:19:12.638505 29653 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 506475378344442183192042916610d5: Neither blocks nor log segments found. Creating new log.
I20260812 01:19:12.638820 29653 log.cc:824] T 00000000000000000000000000000000 P 506475378344442183192042916610d5: Log is configured to *not* fsync() on all Append() calls
I20260812 01:19:12.639559 29653 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 506475378344442183192042916610d5: No bootstrap required, opened a new log
I20260812 01:19:12.641390 29653 raft_consensus.cc:348] T 00000000000000000000000000000000 P 506475378344442183192042916610d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506475378344442183192042916610d5" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 36597 } }
I20260812 01:19:12.641547 29653 raft_consensus.cc:374] T 00000000000000000000000000000000 P 506475378344442183192042916610d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:19:12.641595 29653 raft_consensus.cc:729] T 00000000000000000000000000000000 P 506475378344442183192042916610d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 506475378344442183192042916610d5, State: Initialized, Role: FOLLOWER
I20260812 01:19:12.641741 29653 consensus_queue.cc:260] T 00000000000000000000000000000000 P 506475378344442183192042916610d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506475378344442183192042916610d5" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 36597 } }
I20260812 01:19:12.641850 29653 raft_consensus.cc:388] T 00000000000000000000000000000000 P 506475378344442183192042916610d5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:19:12.641887 29653 raft_consensus.cc:482] T 00000000000000000000000000000000 P 506475378344442183192042916610d5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:19:12.641942 29653 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 506475378344442183192042916610d5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:19:12.642717 29653 raft_consensus.cc:504] T 00000000000000000000000000000000 P 506475378344442183192042916610d5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506475378344442183192042916610d5" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 36597 } }
I20260812 01:19:12.642843 29653 leader_election.cc:302] T 00000000000000000000000000000000 P 506475378344442183192042916610d5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 506475378344442183192042916610d5; no voters:
I20260812 01:19:12.643035 29653 leader_election.cc:288] T 00000000000000000000000000000000 P 506475378344442183192042916610d5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 01:19:12.643059 29665 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 506475378344442183192042916610d5 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:19:12.643178 29665 raft_consensus.cc:686] T 00000000000000000000000000000000 P 506475378344442183192042916610d5 [term 1 LEADER]: Becoming Leader. State: Replica: 506475378344442183192042916610d5, State: Running, Role: LEADER
I20260812 01:19:12.643309 29653 sys_catalog.cc:564] T 00000000000000000000000000000000 P 506475378344442183192042916610d5 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:19:12.643357 29665 consensus_queue.cc:237] T 00000000000000000000000000000000 P 506475378344442183192042916610d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506475378344442183192042916610d5" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 36597 } }
Aug 12 01:19:12 dist-test-slave-t3q3 krb5kdc[29509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497552, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
I20260812 01:19:12.643750 29666 sys_catalog.cc:455] T 00000000000000000000000000000000 P 506475378344442183192042916610d5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "506475378344442183192042916610d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506475378344442183192042916610d5" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 36597 } } }
I20260812 01:19:12.643837 29666 sys_catalog.cc:458] T 00000000000000000000000000000000 P 506475378344442183192042916610d5 [sys.catalog]: This master's current role is: LEADER
I20260812 01:19:12.644035 29668 sys_catalog.cc:455] T 00000000000000000000000000000000 P 506475378344442183192042916610d5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 506475378344442183192042916610d5. Latest consensus state: current_term: 1 leader_uuid: "506475378344442183192042916610d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506475378344442183192042916610d5" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 36597 } } }
I20260812 01:19:12.644099 29672 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:19:12.644131 29668 sys_catalog.cc:458] T 00000000000000000000000000000000 P 506475378344442183192042916610d5 [sys.catalog]: This master's current role is: LEADER
I20260812 01:19:12.644667 29672 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:19:12.646503 29672 catalog_manager.cc:1347] Generated new cluster ID: 3188b8656ebe484dac06133cbe54ee24
I20260812 01:19:12.646557 29672 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 01:19:12.648010 29663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:12.637126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59614 (local address 127.24.0.190:36597)
0812 01:19:12.637325 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:12.637336 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:12.637373 (+ 37us) server_negotiation.cc:407] Connection header received
0812 01:19:12.637905 (+ 532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:12.637915 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:12.637974 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:12.638134 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:12.638756 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:12.639894 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:12.640722 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:12.641132 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:12.641266 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:12.644232 (+ 2966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:12.644237 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:12.644237 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:12.644251 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:12.646365 (+ 2114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:12.646816 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:12.646819 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:12.646819 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:12.646920 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:12.647151 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:12.647152 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:12.647153 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:12.647388 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:12.647547 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:12.647656 (+ 109us) server_negotiation.cc:299] Negotiation successful
0812 01:19:12.647809 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":82,"threads_started":1}
I20260812 01:19:12.652436 29672 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:19:12.652931 29672 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:19:12.662572 29672 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 506475378344442183192042916610d5: Generated new TSK 0
I20260812 01:19:12.662760 29672 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.0.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.0.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.0.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.129.keytab.
Entry for principal kudu/127.24.0.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.129.keytab.
I20260812 01:19:12.694808 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786497534195564-24578-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786497534195564-24578-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786497534195564-24578-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786497534195564-24578-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.0.129:0
--local_ip_for_outbound_sockets=127.24.0.129
--webserver_interface=127.24.0.129
--webserver_port=0
--tserver_master_addrs=127.24.0.190:36597
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.129.keytab
--principal=kudu/127.24.0.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:36551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:19:12.782047 29699 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:19:12.782256 29699 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:19:12.782281 29699 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:19:12.783368 29699 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:19:12.783402 29699 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:19:12.783428 29699 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.0.129
I20260812 01:19:12.784809 29699 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:36551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786497534195564-24578-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786497534195564-24578-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.129.keytab
--principal=kudu/127.24.0.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.0.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786497534195564-24578-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.0.129
--webserver_port=0
--tserver_master_addrs=127.24.0.190:36597
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29699
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.0.129
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786497534195564-24578-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:19:12.784998 29699 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:19:12.785176 29699 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:19:12.790136 29706 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:12.790218 29705 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:12.790153 29708 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:19:12.790314 29699 server_base.cc:1034] running on GCE node
I20260812 01:19:12.790621 29699 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:19:12.790825 29699 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:19:12.791993 29699 hybrid_clock.cc:648] HybridClock initialized: now 1786497552791982 us; error 29 us; skew 500 ppm
Aug 12 01:19:12 dist-test-slave-t3q3 krb5kdc[29509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497552, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:19:12.794404 29699 init.cc:377] Logged in from keytab as kudu/127.24.0.129@KRBTEST.COM (short username kudu)
I20260812 01:19:12.794821 29699 webserver.cc:460] Webserver started at http://127.24.0.129:38165/ using document root <none> and password file <none>
I20260812 01:19:12.795060 29699 fs_manager.cc:359] Metadata directory not provided
I20260812 01:19:12.795128 29699 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:19:12.795287 29699 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:19:12.796375 29699 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786497534195564-24578-0/minicluster-data/ts-0/data/instance:
uuid: "fb7d052467204d11846ca5f35bbe96a3"
format_stamp: "Formatted at 2026-08-12 01:19:12 on dist-test-slave-t3q3"
server_key: "8249f97844b5b8793057b56c9c7fa532"
server_key_iv: "68d798ace4b93ebe45120651127a0542"
server_key_version: "encryptionkey@0"
I20260812 01:19:12.796727 29699 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786497534195564-24578-0/minicluster-data/ts-0/wal/instance:
uuid: "fb7d052467204d11846ca5f35bbe96a3"
format_stamp: "Formatted at 2026-08-12 01:19:12 on dist-test-slave-t3q3"
server_key: "8249f97844b5b8793057b56c9c7fa532"
server_key_iv: "68d798ace4b93ebe45120651127a0542"
server_key_version: "encryptionkey@0"
I20260812 01:19:12.798135 29699 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:19:12.798846 29720 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:12.799007 29699 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 01:19:12.799100 29699 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786497534195564-24578-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786497534195564-24578-0/minicluster-data/ts-0/wal
uuid: "fb7d052467204d11846ca5f35bbe96a3"
format_stamp: "Formatted at 2026-08-12 01:19:12 on dist-test-slave-t3q3"
server_key: "8249f97844b5b8793057b56c9c7fa532"
server_key_iv: "68d798ace4b93ebe45120651127a0542"
server_key_version: "encryptionkey@0"
I20260812 01:19:12.799185 29699 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786497534195564-24578-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786497534195564-24578-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786497534195564-24578-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:19:12.807400 29699 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:19:12.809175 29699 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:19:12.809346 29699 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:19:12.809583 29699 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:19:12.809891 29699 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:19:12.809945 29699 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:12.809986 29699 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:19:12.810017 29699 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:12.816257 29699 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.129:46137
I20260812 01:19:12.816310 29895 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.129:46137 every 8 connection(s)
I20260812 01:19:12.816658 29699 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786497534195564-24578-0/minicluster-data/ts-0/data/info.pb
I20260812 01:19:12.820209 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 29699
I20260812 01:19:12.820322 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786497534195564-24578-0/minicluster-data/ts-0/wal/instance
I20260812 01:19:12.820519 24578 external_mini_cluster.cc:1428] Setting key a863d3526e9f92531a7d9f46b6558f18
Aug 12 01:19:12 dist-test-slave-t3q3 krb5kdc[29509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497552, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.129@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
I20260812 01:19:12.826849 29663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:12.817385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.129:44541 (local address 127.24.0.190:36597)
0812 01:19:12.817471 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:12.817473 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:12.817644 (+ 171us) server_negotiation.cc:407] Connection header received
0812 01:19:12.818097 (+ 453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:12.818099 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:12.818108 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:12.818160 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:12.819045 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:12.819545 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:12.820955 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:12.821251 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:12.821317 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:12.823746 (+ 2429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:12.823748 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:12.823748 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:12.823757 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:12.825443 (+ 1686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:12.825858 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:12.825860 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:12.825860 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:12.825938 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:12.826187 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:12.826188 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:12.826188 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:12.826315 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:12.826398 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:12.826665 (+ 267us) server_negotiation.cc:299] Negotiation successful
0812 01:19:12.826759 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 01:19:12.827054 29899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:12.817338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:36597 (local address 127.24.0.129:44541)
0812 01:19:12.817526 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:12.817561 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:12.818018 (+ 457us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:12.818216 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:12.818219 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:12.818779 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:12.818969 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:12.818973 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:12.819628 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:12.819629 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:12.820862 (+ 1233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:12.820864 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:12.821352 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:12.821353 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:12.821502 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:12.822017 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:12.822027 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:12.823647 (+ 1620us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:19:12.825534 (+ 1887us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:12.825540 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:12.825542 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:12.825775 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:12.826011 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:12.826012 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:12.826012 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:12.826111 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:12.826388 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:19:12.826395 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:19:12.826523 (+ 128us) client_negotiation.cc:769] Sending connection context
0812 01:19:12.826629 (+ 106us) client_negotiation.cc:240] Negotiation successful
0812 01:19:12.826804 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":102,"threads_started":1}
I20260812 01:19:12.827554 29896 heartbeater.cc:347] Connected to a master server at 127.24.0.190:36597
I20260812 01:19:12.827641 29896 heartbeater.cc:464] Registering TS with master...
I20260812 01:19:12.827833 29896 heartbeater.cc:511] Master 127.24.0.190:36597 requested a full tablet report, sending...
I20260812 01:19:12.828414 29588 ts_manager.cc:194] Registered new tserver with Master: fb7d052467204d11846ca5f35bbe96a3 (127.24.0.129:46137)
I20260812 01:19:12.829018 29588 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.0.129@KRBTEST.COM'} at 127.24.0.129:44541
WARNING: no policy specified for kudu/127.24.0.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.0.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.0.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.130.keytab.
Entry for principal kudu/127.24.0.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.130.keytab.
I20260812 01:19:12.847108 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786497534195564-24578-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786497534195564-24578-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786497534195564-24578-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786497534195564-24578-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.0.130:0
--local_ip_for_outbound_sockets=127.24.0.130
--webserver_interface=127.24.0.130
--webserver_port=0
--tserver_master_addrs=127.24.0.190:36597
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.130.keytab
--principal=kudu/127.24.0.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:36551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:19:12.932305 29903 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:19:12.932497 29903 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:19:12.932519 29903 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:19:12.933636 29903 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:19:12.933673 29903 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:19:12.933699 29903 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.0.130
I20260812 01:19:12.935051 29903 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:36551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786497534195564-24578-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786497534195564-24578-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.130.keytab
--principal=kudu/127.24.0.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.0.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786497534195564-24578-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.0.130
--webserver_port=0
--tserver_master_addrs=127.24.0.190:36597
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29903
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.0.130
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786497534195564-24578-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:19:12.935247 29903 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:19:12.935415 29903 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:19:12.940310 29912 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:12.940347 29914 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:19:12.940476 29903 server_base.cc:1034] running on GCE node
W20260812 01:19:12.940358 29911 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:19:12.940725 29903 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:19:12.941004 29903 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:19:12.942166 29903 hybrid_clock.cc:648] HybridClock initialized: now 1786497552942153 us; error 29 us; skew 500 ppm
Aug 12 01:19:12 dist-test-slave-t3q3 krb5kdc[29509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497552, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:19:12.944964 29903 init.cc:377] Logged in from keytab as kudu/127.24.0.130@KRBTEST.COM (short username kudu)
I20260812 01:19:12.945349 29903 webserver.cc:460] Webserver started at http://127.24.0.130:38509/ using document root <none> and password file <none>
I20260812 01:19:12.945536 29903 fs_manager.cc:359] Metadata directory not provided
I20260812 01:19:12.945576 29903 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:19:12.945665 29903 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:19:12.946702 29903 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786497534195564-24578-0/minicluster-data/ts-1/data/instance:
uuid: "8c96ba68d8a8411c9ef79b2672843c36"
format_stamp: "Formatted at 2026-08-12 01:19:12 on dist-test-slave-t3q3"
server_key: "eb09051d2acd9ec47741f318198571c4"
server_key_iv: "74b3db9865a93162dacc7827d0f25d27"
server_key_version: "encryptionkey@0"
I20260812 01:19:12.947003 29903 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786497534195564-24578-0/minicluster-data/ts-1/wal/instance:
uuid: "8c96ba68d8a8411c9ef79b2672843c36"
format_stamp: "Formatted at 2026-08-12 01:19:12 on dist-test-slave-t3q3"
server_key: "eb09051d2acd9ec47741f318198571c4"
server_key_iv: "74b3db9865a93162dacc7827d0f25d27"
server_key_version: "encryptionkey@0"
I20260812 01:19:12.948414 29903 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 01:19:12.949107 29927 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:12.949272 29903 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 01:19:12.949359 29903 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786497534195564-24578-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786497534195564-24578-0/minicluster-data/ts-1/wal
uuid: "8c96ba68d8a8411c9ef79b2672843c36"
format_stamp: "Formatted at 2026-08-12 01:19:12 on dist-test-slave-t3q3"
server_key: "eb09051d2acd9ec47741f318198571c4"
server_key_iv: "74b3db9865a93162dacc7827d0f25d27"
server_key_version: "encryptionkey@0"
I20260812 01:19:12.949429 29903 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786497534195564-24578-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786497534195564-24578-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786497534195564-24578-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:19:12.968633 29903 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:19:12.970687 29903 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:19:12.970841 29903 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:19:12.971096 29903 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:19:12.971441 29903 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:19:12.971498 29903 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:12.971540 29903 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:19:12.971568 29903 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:12.977404 29903 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.130:42223
I20260812 01:19:12.977481 30109 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.130:42223 every 8 connection(s)
I20260812 01:19:12.977902 29903 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786497534195564-24578-0/minicluster-data/ts-1/data/info.pb
I20260812 01:19:12.983029 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 29903
I20260812 01:19:12.983151 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786497534195564-24578-0/minicluster-data/ts-1/wal/instance
I20260812 01:19:12.983353 24578 external_mini_cluster.cc:1428] Setting key c1232f3700e7b4ee5d6bd93233af5bee
Aug 12 01:19:12 dist-test-slave-t3q3 krb5kdc[29509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497552, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.130@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
I20260812 01:19:12.987684 29663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:12.978628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.130:33273 (local address 127.24.0.190:36597)
0812 01:19:12.978688 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:12.978690 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:12.978899 (+ 209us) server_negotiation.cc:407] Connection header received
0812 01:19:12.979431 (+ 532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:12.979432 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:12.979445 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:12.979505 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:12.980537 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:12.981038 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:12.982038 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:12.982247 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:12.982320 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:12.984758 (+ 2438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:12.984759 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:12.984759 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:12.984766 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:12.986047 (+ 1281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:12.986552 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:12.986553 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:12.986554 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:12.986618 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:12.986916 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:12.986916 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:12.986917 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:12.987020 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:12.987074 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:12.987461 (+ 387us) server_negotiation.cc:299] Negotiation successful
0812 01:19:12.987550 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260812 01:19:12.987767 30114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:12.978610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:36597 (local address 127.24.0.130:33273)
0812 01:19:12.978772 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:12.978825 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:12.979324 (+ 499us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:12.979571 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:12.979575 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:12.980238 (+ 663us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:12.980430 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:12.980436 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:12.981137 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:12.981138 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:12.981932 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:12.981934 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:12.982339 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:12.982340 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:12.982459 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:12.982873 (+ 414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:12.982881 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:12.984646 (+ 1765us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:19:12.986113 (+ 1467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:12.986121 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:12.986124 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:12.986463 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:12.986699 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:12.986699 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:12.986700 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:12.986843 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:12.987089 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:19:12.987096 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:19:12.987304 (+ 208us) client_negotiation.cc:769] Sending connection context
0812 01:19:12.987421 (+ 117us) client_negotiation.cc:240] Negotiation successful
0812 01:19:12.987586 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":88,"threads_started":1}
I20260812 01:19:12.988282 30110 heartbeater.cc:347] Connected to a master server at 127.24.0.190:36597
I20260812 01:19:12.988368 30110 heartbeater.cc:464] Registering TS with master...
I20260812 01:19:12.988564 30110 heartbeater.cc:511] Master 127.24.0.190:36597 requested a full tablet report, sending...
I20260812 01:19:12.988973 29588 ts_manager.cc:194] Registered new tserver with Master: 8c96ba68d8a8411c9ef79b2672843c36 (127.24.0.130:42223)
I20260812 01:19:12.989518 29588 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.0.130@KRBTEST.COM'} at 127.24.0.130:33273
WARNING: no policy specified for kudu/127.24.0.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.0.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.0.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.131.keytab.
Entry for principal kudu/127.24.0.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.131.keytab.
I20260812 01:19:13.010532 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786497534195564-24578-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786497534195564-24578-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786497534195564-24578-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786497534195564-24578-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.0.131:0
--local_ip_for_outbound_sockets=127.24.0.131
--webserver_interface=127.24.0.131
--webserver_port=0
--tserver_master_addrs=127.24.0.190:36597
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.131.keytab
--principal=kudu/127.24.0.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:36551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:19:13.093832 30117 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:19:13.094024 30117 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:19:13.094048 30117 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:19:13.095109 30117 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:19:13.095142 30117 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:19:13.095166 30117 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.0.131
I20260812 01:19:13.096513 30117 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:36551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786497534195564-24578-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786497534195564-24578-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.131.keytab
--principal=kudu/127.24.0.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.0.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786497534195564-24578-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.0.131
--webserver_port=0
--tserver_master_addrs=127.24.0.190:36597
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30117
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.0.131
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786497534195564-24578-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:19:13.096688 30117 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:19:13.096863 30117 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:19:13.101794 30130 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:13.101835 30128 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:19:13.101887 30117 server_base.cc:1034] running on GCE node
W20260812 01:19:13.101825 30132 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:19:13.102236 30117 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:19:13.102484 30117 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:19:13.103663 30117 hybrid_clock.cc:648] HybridClock initialized: now 1786497553103651 us; error 31 us; skew 500 ppm
Aug 12 01:19:13 dist-test-slave-t3q3 krb5kdc[29509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497553, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:19:13.106407 30117 init.cc:377] Logged in from keytab as kudu/127.24.0.131@KRBTEST.COM (short username kudu)
I20260812 01:19:13.106904 30117 webserver.cc:460] Webserver started at http://127.24.0.131:33551/ using document root <none> and password file <none>
I20260812 01:19:13.107161 30117 fs_manager.cc:359] Metadata directory not provided
I20260812 01:19:13.107234 30117 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:19:13.107398 30117 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:19:13.108520 30117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786497534195564-24578-0/minicluster-data/ts-2/data/instance:
uuid: "94f5178d5d9346b694d7a3b00f0bb245"
format_stamp: "Formatted at 2026-08-12 01:19:13 on dist-test-slave-t3q3"
server_key: "138ed80ac219b06aeaff5d73de75a07c"
server_key_iv: "fa8dc102ced93c3e30999489404ae2b3"
server_key_version: "encryptionkey@0"
I20260812 01:19:13.108894 30117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786497534195564-24578-0/minicluster-data/ts-2/wal/instance:
uuid: "94f5178d5d9346b694d7a3b00f0bb245"
format_stamp: "Formatted at 2026-08-12 01:19:13 on dist-test-slave-t3q3"
server_key: "138ed80ac219b06aeaff5d73de75a07c"
server_key_iv: "fa8dc102ced93c3e30999489404ae2b3"
server_key_version: "encryptionkey@0"
I20260812 01:19:13.110201 30117 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260812 01:19:13.110914 30146 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:13.111083 30117 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:13.111174 30117 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786497534195564-24578-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786497534195564-24578-0/minicluster-data/ts-2/wal
uuid: "94f5178d5d9346b694d7a3b00f0bb245"
format_stamp: "Formatted at 2026-08-12 01:19:13 on dist-test-slave-t3q3"
server_key: "138ed80ac219b06aeaff5d73de75a07c"
server_key_iv: "fa8dc102ced93c3e30999489404ae2b3"
server_key_version: "encryptionkey@0"
I20260812 01:19:13.111261 30117 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786497534195564-24578-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786497534195564-24578-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786497534195564-24578-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:19:13.124182 30117 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:19:13.125952 30117 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:19:13.126112 30117 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:19:13.126344 30117 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:19:13.126649 30117 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:19:13.126709 30117 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:13.126754 30117 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:19:13.126780 30117 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:13.132973 30117 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.131:33537
I20260812 01:19:13.133026 30328 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.131:33537 every 8 connection(s)
I20260812 01:19:13.133365 30117 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786497534195564-24578-0/minicluster-data/ts-2/data/info.pb
I20260812 01:19:13.136237 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 30117
I20260812 01:19:13.136349 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786497534195564-24578-0/minicluster-data/ts-2/wal/instance
I20260812 01:19:13.136533 24578 external_mini_cluster.cc:1428] Setting key 39a4f220e8339a40c0d57759f45f8a56
Aug 12 01:19:13 dist-test-slave-t3q3 krb5kdc[29509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497553, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.131@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
I20260812 01:19:13.142688 29663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:13.134094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.131:51965 (local address 127.24.0.190:36597)
0812 01:19:13.134172 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:13.134175 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:13.134359 (+ 184us) server_negotiation.cc:407] Connection header received
0812 01:19:13.134818 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:13.134819 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:13.134829 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:13.134883 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:13.135759 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:13.136348 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:13.137727 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:13.137926 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:13.138004 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:13.140097 (+ 2093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:13.140100 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:13.140100 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:13.140112 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:13.141283 (+ 1171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:13.141712 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:13.141714 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:13.141714 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:13.141773 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:13.142039 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:13.142040 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:13.142041 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:13.142152 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:13.142236 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:13.142508 (+ 272us) server_negotiation.cc:299] Negotiation successful
0812 01:19:13.142600 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 01:19:13.142872 30333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:13.134074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:36597 (local address 127.24.0.131:51965)
0812 01:19:13.134239 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:13.134274 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:13.134737 (+ 463us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:13.134935 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:13.134938 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:13.135494 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:13.135681 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:13.135686 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:13.136461 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:13.136462 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:13.137628 (+ 1166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:13.137631 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:13.138012 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:13.138013 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:13.138147 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:13.138596 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:13.138605 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:13.139968 (+ 1363us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:19:13.141379 (+ 1411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:13.141386 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:13.141388 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:13.141613 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:13.141853 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:13.141853 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:13.141854 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:13.141962 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:13.142226 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:19:13.142233 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:19:13.142371 (+ 138us) client_negotiation.cc:769] Sending connection context
0812 01:19:13.142483 (+ 112us) client_negotiation.cc:240] Negotiation successful
0812 01:19:13.142636 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":94,"threads_started":1}
I20260812 01:19:13.143323 30329 heartbeater.cc:347] Connected to a master server at 127.24.0.190:36597
I20260812 01:19:13.143401 30329 heartbeater.cc:464] Registering TS with master...
I20260812 01:19:13.143596 30329 heartbeater.cc:511] Master 127.24.0.190:36597 requested a full tablet report, sending...
I20260812 01:19:13.144008 29588 ts_manager.cc:194] Registered new tserver with Master: 94f5178d5d9346b694d7a3b00f0bb245 (127.24.0.131:33537)
I20260812 01:19:13.144456 29588 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.0.131@KRBTEST.COM'} at 127.24.0.131:51965
I20260812 01:19:13.148347 24578 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 01:19:13 dist-test-slave-t3q3 krb5kdc[29509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497553, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 12 01:19:13 dist-test-slave-t3q3 krb5kdc[29509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497553, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
I20260812 01:19:13.172155 29663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:13.165060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59618 (local address 127.24.0.190:36597)
0812 01:19:13.165141 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:13.165143 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:13.165252 (+ 109us) server_negotiation.cc:407] Connection header received
0812 01:19:13.165340 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:13.165343 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:13.165357 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:13.165415 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:13.166028 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:13.166517 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:13.167290 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:13.167483 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:13.167559 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:13.169750 (+ 2191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:13.169752 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:13.169753 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:13.169761 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:13.170944 (+ 1183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:13.171300 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:13.171301 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:13.171302 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:13.171363 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:13.171573 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:13.171574 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:13.171574 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:13.171727 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:13.171843 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:13.171942 (+ 99us) server_negotiation.cc:299] Negotiation successful
0812 01:19:13.172027 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
W20260812 01:19:13.172416 29588 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:59618
I20260812 01:19:13.172881 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 29699
I20260812 01:19:13.197810 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 29903
I20260812 01:19:13.205714 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 30117
I20260812 01:19:13.227046 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 29532
2026-08-12T01:19:13Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2133 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786497534195564-24578-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 01:19:13 dist-test-slave-t3q3 krb5kdc[30350](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 01:19:13 dist-test-slave-t3q3 krb5kdc[30350](info): set up 2 sockets
Aug 12 01:19:13 dist-test-slave-t3q3 krb5kdc[30350](info): commencing operation
krb5kdc: starting...
W20260812 01:19:14.505705 24578 mini_kdc.cc:121] Time spent starting KDC: real 1.255s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786497534195564-24578-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786497534195564-24578-0/krb5kdc/test-user.keytab.
Aug 12 01:19:14 dist-test-slave-t3q3 krb5kdc[30350](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497554, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T01:19:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:19:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.0.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.0.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.0.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab.
Entry for principal kudu/127.24.0.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab.
I20260812 01:19:14.631033 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786497534195564-24578-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786497534195564-24578-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786497534195564-24578-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.190:42255
--webserver_interface=127.24.0.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab
--principal=kudu/127.24.0.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:44633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.0.190:42255
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:19:14.713738 30367 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:19:14.713981 30367 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:19:14.714018 30367 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:19:14.715147 30367 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:19:14.715217 30367 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:19:14.715241 30367 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:19:14.715261 30367 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:19:14.715291 30367 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:19:14.716835 30367 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:44633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786497534195564-24578-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786497534195564-24578-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.0.190:42255
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab
--principal=kudu/127.24.0.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.190:42255
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.0.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30367
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786497534195564-24578-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:19:14.717120 30367 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:19:14.717346 30367 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:19:14.722157 30378 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:14.722132 30380 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:14.722132 30374 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:19:14.722625 30367 server_base.cc:1034] running on GCE node
I20260812 01:19:14.722918 30367 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:19:14.723197 30367 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:19:14.724362 30367 hybrid_clock.cc:648] HybridClock initialized: now 1786497554724340 us; error 35 us; skew 500 ppm
Aug 12 01:19:14 dist-test-slave-t3q3 krb5kdc[30350](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497554, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:19:14.731374 30367 init.cc:377] Logged in from keytab as kudu/127.24.0.190@KRBTEST.COM (short username kudu)
I20260812 01:19:14.731940 30367 webserver.cc:460] Webserver started at http://127.24.0.190:46161/ using document root <none> and password file <none>
I20260812 01:19:14.732260 30367 fs_manager.cc:359] Metadata directory not provided
I20260812 01:19:14.732342 30367 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:19:14.732487 30367 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:19:14.733749 30367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786497534195564-24578-0/minicluster-data/master-0/data/instance:
uuid: "0bf84dc95d1e4b5eba56f52c32c2a3d5"
format_stamp: "Formatted at 2026-08-12 01:19:14 on dist-test-slave-t3q3"
server_key: "74b22f3c975b0d4f55659a61d63f77d6"
server_key_iv: "7114bf927cd59043bb19e242b82a60ae"
server_key_version: "encryptionkey@0"
I20260812 01:19:14.734179 30367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786497534195564-24578-0/minicluster-data/master-0/wal/instance:
uuid: "0bf84dc95d1e4b5eba56f52c32c2a3d5"
format_stamp: "Formatted at 2026-08-12 01:19:14 on dist-test-slave-t3q3"
server_key: "74b22f3c975b0d4f55659a61d63f77d6"
server_key_iv: "7114bf927cd59043bb19e242b82a60ae"
server_key_version: "encryptionkey@0"
I20260812 01:19:14.736045 30367 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260812 01:19:14.736835 30392 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:14.737012 30367 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 01:19:14.737118 30367 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786497534195564-24578-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786497534195564-24578-0/minicluster-data/master-0/wal
uuid: "0bf84dc95d1e4b5eba56f52c32c2a3d5"
format_stamp: "Formatted at 2026-08-12 01:19:14 on dist-test-slave-t3q3"
server_key: "74b22f3c975b0d4f55659a61d63f77d6"
server_key_iv: "7114bf927cd59043bb19e242b82a60ae"
server_key_version: "encryptionkey@0"
I20260812 01:19:14.737221 30367 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786497534195564-24578-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786497534195564-24578-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786497534195564-24578-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:19:14.755663 30367 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:19:14.757551 30367 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:19:14.757722 30367 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:19:14.762013 30367 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.190:42255
I20260812 01:19:14.762066 30480 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.190:42255 every 8 connection(s)
I20260812 01:19:14.762548 30367 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
I20260812 01:19:14.763236 30481 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:19:14.765551 30481 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0bf84dc95d1e4b5eba56f52c32c2a3d5: Bootstrap starting.
I20260812 01:19:14.766167 30481 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0bf84dc95d1e4b5eba56f52c32c2a3d5: Neither blocks nor log segments found. Creating new log.
I20260812 01:19:14.766489 30481 log.cc:824] T 00000000000000000000000000000000 P 0bf84dc95d1e4b5eba56f52c32c2a3d5: Log is configured to *not* fsync() on all Append() calls
I20260812 01:19:14.766821 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 30367
I20260812 01:19:14.766943 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786497534195564-24578-0/minicluster-data/master-0/wal/instance
I20260812 01:19:14.767130 24578 external_mini_cluster.cc:1428] Setting key 5e980516bd7127657f4fb04bfc155dfc
I20260812 01:19:14.767352 30481 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0bf84dc95d1e4b5eba56f52c32c2a3d5: No bootstrap required, opened a new log
I20260812 01:19:14.769737 30481 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0bf84dc95d1e4b5eba56f52c32c2a3d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bf84dc95d1e4b5eba56f52c32c2a3d5" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 42255 } }
I20260812 01:19:14.769905 30481 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0bf84dc95d1e4b5eba56f52c32c2a3d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:19:14.769948 30481 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0bf84dc95d1e4b5eba56f52c32c2a3d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bf84dc95d1e4b5eba56f52c32c2a3d5, State: Initialized, Role: FOLLOWER
I20260812 01:19:14.770130 30481 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0bf84dc95d1e4b5eba56f52c32c2a3d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bf84dc95d1e4b5eba56f52c32c2a3d5" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 42255 } }
I20260812 01:19:14.770227 30481 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0bf84dc95d1e4b5eba56f52c32c2a3d5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:19:14.770264 30481 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0bf84dc95d1e4b5eba56f52c32c2a3d5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:19:14.770315 30481 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0bf84dc95d1e4b5eba56f52c32c2a3d5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:19:14.771082 30481 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0bf84dc95d1e4b5eba56f52c32c2a3d5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bf84dc95d1e4b5eba56f52c32c2a3d5" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 42255 } }
I20260812 01:19:14.771175 30481 leader_election.cc:302] T 00000000000000000000000000000000 P 0bf84dc95d1e4b5eba56f52c32c2a3d5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0bf84dc95d1e4b5eba56f52c32c2a3d5; no voters:
I20260812 01:19:14.771348 30481 leader_election.cc:288] T 00000000000000000000000000000000 P 0bf84dc95d1e4b5eba56f52c32c2a3d5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 01:19:14.771379 30487 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0bf84dc95d1e4b5eba56f52c32c2a3d5 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:19:14.771486 30487 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0bf84dc95d1e4b5eba56f52c32c2a3d5 [term 1 LEADER]: Becoming Leader. State: Replica: 0bf84dc95d1e4b5eba56f52c32c2a3d5, State: Running, Role: LEADER
I20260812 01:19:14.771569 30487 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0bf84dc95d1e4b5eba56f52c32c2a3d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bf84dc95d1e4b5eba56f52c32c2a3d5" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 42255 } }
I20260812 01:19:14.771621 30481 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0bf84dc95d1e4b5eba56f52c32c2a3d5 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:19:14.771900 30487 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bf84dc95d1e4b5eba56f52c32c2a3d5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0bf84dc95d1e4b5eba56f52c32c2a3d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bf84dc95d1e4b5eba56f52c32c2a3d5" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 42255 } } }
I20260812 01:19:14.772099 30487 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bf84dc95d1e4b5eba56f52c32c2a3d5 [sys.catalog]: This master's current role is: LEADER
I20260812 01:19:14.772308 30487 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bf84dc95d1e4b5eba56f52c32c2a3d5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0bf84dc95d1e4b5eba56f52c32c2a3d5. Latest consensus state: current_term: 1 leader_uuid: "0bf84dc95d1e4b5eba56f52c32c2a3d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bf84dc95d1e4b5eba56f52c32c2a3d5" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 42255 } } }
I20260812 01:19:14.772423 30487 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bf84dc95d1e4b5eba56f52c32c2a3d5 [sys.catalog]: This master's current role is: LEADER
I20260812 01:19:14.772784 30496 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:19:14.773392 30496 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:19:14.775161 30496 catalog_manager.cc:1347] Generated new cluster ID: 1ed131f038934ab09fec8dfff27b1f59
I20260812 01:19:14.775218 30496 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 01:19:14 dist-test-slave-t3q3 krb5kdc[30350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497554, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
I20260812 01:19:14.781455 30486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:14.767690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41674 (local address 127.24.0.190:42255)
0812 01:19:14.767894 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:14.767903 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:14.767936 (+ 33us) server_negotiation.cc:407] Connection header received
0812 01:19:14.769717 (+ 1781us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:14.769724 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:14.769774 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:14.769891 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:14.770800 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:14.772029 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:14.774048 (+ 2019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:14.774580 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:14.774702 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:14.777562 (+ 2860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:14.777567 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:14.777567 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:14.777579 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:14.779573 (+ 1994us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:14.779963 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:14.779966 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:14.779966 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:14.780116 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:14.780339 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:14.780340 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:14.780340 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:14.780689 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:14.780898 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:14.780981 (+ 83us) server_negotiation.cc:299] Negotiation successful
0812 01:19:14.781258 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":113,"threads_started":1}
I20260812 01:19:14.789284 30496 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:19:14.789817 30496 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:19:14.792624 30496 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0bf84dc95d1e4b5eba56f52c32c2a3d5: Generated new TSK 0
I20260812 01:19:14.792795 30496 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.0.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.0.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.0.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.129.keytab.
Entry for principal kudu/127.24.0.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.129.keytab.
I20260812 01:19:14.826502 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786497534195564-24578-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786497534195564-24578-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786497534195564-24578-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786497534195564-24578-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.0.129:0
--local_ip_for_outbound_sockets=127.24.0.129
--webserver_interface=127.24.0.129
--webserver_port=0
--tserver_master_addrs=127.24.0.190:42255
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.129.keytab
--principal=kudu/127.24.0.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:44633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:19:14.947785 30519 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:19:14.948078 30519 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:19:14.948114 30519 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:19:14.949793 30519 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:19:14.949846 30519 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:19:14.949884 30519 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.0.129
I20260812 01:19:14.952124 30519 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:44633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786497534195564-24578-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786497534195564-24578-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.129.keytab
--principal=kudu/127.24.0.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.0.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786497534195564-24578-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.0.129
--webserver_port=0
--tserver_master_addrs=127.24.0.190:42255
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30519
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.0.129
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786497534195564-24578-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:19:14.952375 30519 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:19:14.952620 30519 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:19:14.958204 30532 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:14.958236 30531 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:14.958220 30534 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:19:14.958451 30519 server_base.cc:1034] running on GCE node
I20260812 01:19:14.958701 30519 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:19:14.958947 30519 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:19:14.960151 30519 hybrid_clock.cc:648] HybridClock initialized: now 1786497554960121 us; error 37 us; skew 500 ppm
Aug 12 01:19:14 dist-test-slave-t3q3 krb5kdc[30350](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497554, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:19:14.962900 30519 init.cc:377] Logged in from keytab as kudu/127.24.0.129@KRBTEST.COM (short username kudu)
I20260812 01:19:14.963310 30519 webserver.cc:460] Webserver started at http://127.24.0.129:46379/ using document root <none> and password file <none>
I20260812 01:19:14.963495 30519 fs_manager.cc:359] Metadata directory not provided
I20260812 01:19:14.963536 30519 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:19:14.963625 30519 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:19:14.964732 30519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786497534195564-24578-0/minicluster-data/ts-0/data/instance:
uuid: "e9ab3d074ccb43fb8ad8282a5e7327e8"
format_stamp: "Formatted at 2026-08-12 01:19:14 on dist-test-slave-t3q3"
server_key: "5cf120a24152725fd81ef346727e57ca"
server_key_iv: "e1785cd5a76daedbd58f43c20117b456"
server_key_version: "encryptionkey@0"
I20260812 01:19:14.965101 30519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786497534195564-24578-0/minicluster-data/ts-0/wal/instance:
uuid: "e9ab3d074ccb43fb8ad8282a5e7327e8"
format_stamp: "Formatted at 2026-08-12 01:19:14 on dist-test-slave-t3q3"
server_key: "5cf120a24152725fd81ef346727e57ca"
server_key_iv: "e1785cd5a76daedbd58f43c20117b456"
server_key_version: "encryptionkey@0"
I20260812 01:19:14.966650 30519 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 01:19:14.967464 30543 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:14.967648 30519 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:19:14.967768 30519 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786497534195564-24578-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786497534195564-24578-0/minicluster-data/ts-0/wal
uuid: "e9ab3d074ccb43fb8ad8282a5e7327e8"
format_stamp: "Formatted at 2026-08-12 01:19:14 on dist-test-slave-t3q3"
server_key: "5cf120a24152725fd81ef346727e57ca"
server_key_iv: "e1785cd5a76daedbd58f43c20117b456"
server_key_version: "encryptionkey@0"
I20260812 01:19:14.967854 30519 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786497534195564-24578-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786497534195564-24578-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786497534195564-24578-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:19:14.998349 30519 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:19:15.000953 30519 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:19:15.001147 30519 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:19:15.001426 30519 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:19:15.001784 30519 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:19:15.001842 30519 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:15.001884 30519 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:19:15.001929 30519 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:15.007733 30519 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.129:37117
I20260812 01:19:15.007841 30731 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.129:37117 every 8 connection(s)
I20260812 01:19:15.008188 30519 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786497534195564-24578-0/minicluster-data/ts-0/data/info.pb
I20260812 01:19:15.009861 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 30519
I20260812 01:19:15.009974 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786497534195564-24578-0/minicluster-data/ts-0/wal/instance
I20260812 01:19:15.010159 24578 external_mini_cluster.cc:1428] Setting key 76db0a886b785875f234d96c58547de0
Aug 12 01:19:15 dist-test-slave-t3q3 krb5kdc[30350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497554, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.129@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
I20260812 01:19:15.018168 30486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:15.009001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.129:55325 (local address 127.24.0.190:42255)
0812 01:19:15.009067 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:15.009069 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:15.009310 (+ 241us) server_negotiation.cc:407] Connection header received
0812 01:19:15.010328 (+ 1018us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:15.010329 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:15.010347 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:15.010417 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:15.011090 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:15.011588 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:15.012682 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:15.012896 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:15.012975 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:15.015176 (+ 2201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:15.015178 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:15.015178 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:15.015195 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:15.016700 (+ 1505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:15.017144 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:15.017146 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:15.017146 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:15.017210 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:15.017469 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:15.017471 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:15.017471 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:15.017589 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:15.017672 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:15.017960 (+ 288us) server_negotiation.cc:299] Negotiation successful
0812 01:19:15.018062 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 01:19:15.018337 30740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:15.009020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:42255 (local address 127.24.0.129:55325)
0812 01:19:15.009180 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:15.009216 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:15.010206 (+ 990us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:15.010535 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:15.010538 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:15.010792 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:15.010983 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:15.010987 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:15.011710 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:15.011711 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:15.012579 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:15.012581 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:15.013008 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:15.013009 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:15.013141 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:15.013692 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:15.013701 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:15.015058 (+ 1357us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:19:15.016806 (+ 1748us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:15.016814 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:15.016816 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:15.017048 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:15.017287 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:15.017288 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:15.017288 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:15.017400 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:15.017669 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:19:15.017677 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:19:15.017819 (+ 142us) client_negotiation.cc:769] Sending connection context
0812 01:19:15.017923 (+ 104us) client_negotiation.cc:240] Negotiation successful
0812 01:19:15.018086 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":74,"threads_started":1}
I20260812 01:19:15.018828 30738 heartbeater.cc:347] Connected to a master server at 127.24.0.190:42255
I20260812 01:19:15.018954 30738 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.24.0.130@KRBTEST.COM; defaulting to no policy
I20260812 01:19:15.019168 30738 heartbeater.cc:511] Master 127.24.0.190:42255 requested a full tablet report, sending...
I20260812 01:19:15.019757 30424 ts_manager.cc:194] Registered new tserver with Master: e9ab3d074ccb43fb8ad8282a5e7327e8 (127.24.0.129:37117)
I20260812 01:19:15.020462 30424 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.0.129@KRBTEST.COM'} at 127.24.0.129:55325
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.0.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.0.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.130.keytab.
Entry for principal kudu/127.24.0.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.130.keytab.
I20260812 01:19:15.038439 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786497534195564-24578-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786497534195564-24578-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786497534195564-24578-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786497534195564-24578-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.0.130:0
--local_ip_for_outbound_sockets=127.24.0.130
--webserver_interface=127.24.0.130
--webserver_port=0
--tserver_master_addrs=127.24.0.190:42255
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.130.keytab
--principal=kudu/127.24.0.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:44633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:19:15.122679 30745 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:19:15.122870 30745 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:19:15.122893 30745 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:19:15.124135 30745 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:19:15.124194 30745 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:19:15.124223 30745 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.0.130
I20260812 01:19:15.125730 30745 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:44633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786497534195564-24578-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786497534195564-24578-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.130.keytab
--principal=kudu/127.24.0.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.0.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786497534195564-24578-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.0.130
--webserver_port=0
--tserver_master_addrs=127.24.0.190:42255
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30745
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.0.130
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786497534195564-24578-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:19:15.125974 30745 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:19:15.126214 30745 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:19:15.131103 30753 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:19:15.131176 30745 server_base.cc:1034] running on GCE node
W20260812 01:19:15.131140 30751 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:15.131110 30750 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:19:15.131603 30745 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:19:15.131870 30745 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:19:15.133033 30745 hybrid_clock.cc:648] HybridClock initialized: now 1786497555133008 us; error 60 us; skew 500 ppm
Aug 12 01:19:15 dist-test-slave-t3q3 krb5kdc[30350](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497555, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:19:15.135686 30745 init.cc:377] Logged in from keytab as kudu/127.24.0.130@KRBTEST.COM (short username kudu)
I20260812 01:19:15.136308 30745 webserver.cc:460] Webserver started at http://127.24.0.130:45787/ using document root <none> and password file <none>
I20260812 01:19:15.136554 30745 fs_manager.cc:359] Metadata directory not provided
I20260812 01:19:15.136631 30745 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:19:15.136783 30745 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:19:15.137784 30745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786497534195564-24578-0/minicluster-data/ts-1/data/instance:
uuid: "c1d1fa3a12f34963a60b1991a4d2e752"
format_stamp: "Formatted at 2026-08-12 01:19:15 on dist-test-slave-t3q3"
server_key: "4cc010a3e256192ad72c39324ceb1d7b"
server_key_iv: "ed9940397e82351427db359d6bf228a7"
server_key_version: "encryptionkey@0"
I20260812 01:19:15.138151 30745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786497534195564-24578-0/minicluster-data/ts-1/wal/instance:
uuid: "c1d1fa3a12f34963a60b1991a4d2e752"
format_stamp: "Formatted at 2026-08-12 01:19:15 on dist-test-slave-t3q3"
server_key: "4cc010a3e256192ad72c39324ceb1d7b"
server_key_iv: "ed9940397e82351427db359d6bf228a7"
server_key_version: "encryptionkey@0"
I20260812 01:19:15.139461 30745 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 01:19:15.140290 30761 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:15.140479 30745 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:19:15.140586 30745 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786497534195564-24578-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786497534195564-24578-0/minicluster-data/ts-1/wal
uuid: "c1d1fa3a12f34963a60b1991a4d2e752"
format_stamp: "Formatted at 2026-08-12 01:19:15 on dist-test-slave-t3q3"
server_key: "4cc010a3e256192ad72c39324ceb1d7b"
server_key_iv: "ed9940397e82351427db359d6bf228a7"
server_key_version: "encryptionkey@0"
I20260812 01:19:15.140661 30745 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786497534195564-24578-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786497534195564-24578-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786497534195564-24578-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:19:15.164608 30745 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:19:15.166566 30745 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:19:15.166733 30745 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:19:15.166982 30745 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:19:15.167304 30745 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:19:15.167361 30745 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:15.167407 30745 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:19:15.167433 30745 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:15.173980 30745 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.130:40939
I20260812 01:19:15.174026 30945 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.130:40939 every 8 connection(s)
I20260812 01:19:15.174353 30745 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786497534195564-24578-0/minicluster-data/ts-1/data/info.pb
I20260812 01:19:15.174670 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 30745
I20260812 01:19:15.174780 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786497534195564-24578-0/minicluster-data/ts-1/wal/instance
I20260812 01:19:15.174999 24578 external_mini_cluster.cc:1428] Setting key 66ea3a89c87c3300fd06131866c13751
Aug 12 01:19:15 dist-test-slave-t3q3 krb5kdc[30350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497555, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.130@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
I20260812 01:19:15.184520 30486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:15.175110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.130:40547 (local address 127.24.0.190:42255)
0812 01:19:15.175157 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:15.175160 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:15.175410 (+ 250us) server_negotiation.cc:407] Connection header received
0812 01:19:15.176469 (+ 1059us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:15.176471 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:15.176485 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:15.176555 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:15.177170 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:15.177847 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:15.179167 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:15.179348 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:15.179431 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:15.181613 (+ 2182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:15.181615 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:15.181616 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:15.181624 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:15.183013 (+ 1389us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:15.183443 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:15.183446 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:15.183446 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:15.183508 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:15.183777 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:15.183778 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:15.183778 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:15.183899 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:15.183977 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:15.184341 (+ 364us) server_negotiation.cc:299] Negotiation successful
0812 01:19:15.184431 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
WARNING: no policy specified for kudu/127.24.0.131@KRBTEST.COM; defaulting to no policy
I20260812 01:19:15.184721 30949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:15.175091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:42255 (local address 127.24.0.130:40547)
0812 01:19:15.175279 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:15.175329 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:15.176367 (+ 1038us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:15.176619 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:15.176622 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:15.176874 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:15.177058 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:15.177062 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:15.177942 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:15.177944 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:15.179089 (+ 1145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:15.179092 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:15.179440 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:15.179441 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:15.179575 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:15.180137 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:15.180146 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:15.181515 (+ 1369us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:19:15.183106 (+ 1591us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:15.183113 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:15.183115 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:15.183347 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:15.183579 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:15.183580 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:15.183580 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:15.183702 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:15.183983 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:19:15.183990 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:19:15.184198 (+ 208us) client_negotiation.cc:769] Sending connection context
0812 01:19:15.184302 (+ 104us) client_negotiation.cc:240] Negotiation successful
0812 01:19:15.184462 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":93,"threads_started":1}
I20260812 01:19:15.185200 30946 heartbeater.cc:347] Connected to a master server at 127.24.0.190:42255
I20260812 01:19:15.185288 30946 heartbeater.cc:464] Registering TS with master...
I20260812 01:19:15.185427 30946 heartbeater.cc:511] Master 127.24.0.190:42255 requested a full tablet report, sending...
I20260812 01:19:15.185891 30424 ts_manager.cc:194] Registered new tserver with Master: c1d1fa3a12f34963a60b1991a4d2e752 (127.24.0.130:40939)
I20260812 01:19:15.186280 30424 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.0.130@KRBTEST.COM'} at 127.24.0.130:40547
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.0.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.0.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.131.keytab.
Entry for principal kudu/127.24.0.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.131.keytab.
I20260812 01:19:15.202920 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786497534195564-24578-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786497534195564-24578-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786497534195564-24578-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786497534195564-24578-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.0.131:0
--local_ip_for_outbound_sockets=127.24.0.131
--webserver_interface=127.24.0.131
--webserver_port=0
--tserver_master_addrs=127.24.0.190:42255
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.131.keytab
--principal=kudu/127.24.0.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:44633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:19:15.288955 30954 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:19:15.289171 30954 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:19:15.289196 30954 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:19:15.290311 30954 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:19:15.290350 30954 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:19:15.290376 30954 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.0.131
I20260812 01:19:15.291841 30954 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:44633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786497534195564-24578-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786497534195564-24578-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.131.keytab
--principal=kudu/127.24.0.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.0.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786497534195564-24578-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.0.131
--webserver_port=0
--tserver_master_addrs=127.24.0.190:42255
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30954
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.0.131
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786497534195564-24578-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:19:15.292030 30954 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:19:15.292287 30954 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:19:15.297498 30964 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:15.297514 30960 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:15.297524 30962 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:19:15.297845 30954 server_base.cc:1034] running on GCE node
I20260812 01:19:15.298130 30954 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:19:15.298388 30954 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:19:15.299572 30954 hybrid_clock.cc:648] HybridClock initialized: now 1786497555299565 us; error 37 us; skew 500 ppm
Aug 12 01:19:15 dist-test-slave-t3q3 krb5kdc[30350](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497555, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:19:15.302309 30954 init.cc:377] Logged in from keytab as kudu/127.24.0.131@KRBTEST.COM (short username kudu)
I20260812 01:19:15.302780 30954 webserver.cc:460] Webserver started at http://127.24.0.131:33215/ using document root <none> and password file <none>
I20260812 01:19:15.303014 30954 fs_manager.cc:359] Metadata directory not provided
I20260812 01:19:15.303084 30954 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:19:15.303246 30954 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:19:15.304414 30954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786497534195564-24578-0/minicluster-data/ts-2/data/instance:
uuid: "69c1f4ab561d4e84bb9ee86d605b1d2c"
format_stamp: "Formatted at 2026-08-12 01:19:15 on dist-test-slave-t3q3"
server_key: "1f5ebb90b9938bd92cec3a818f2cad7d"
server_key_iv: "b2ab9cd9eca7f449e21de9e14d2979f7"
server_key_version: "encryptionkey@0"
I20260812 01:19:15.304802 30954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786497534195564-24578-0/minicluster-data/ts-2/wal/instance:
uuid: "69c1f4ab561d4e84bb9ee86d605b1d2c"
format_stamp: "Formatted at 2026-08-12 01:19:15 on dist-test-slave-t3q3"
server_key: "1f5ebb90b9938bd92cec3a818f2cad7d"
server_key_iv: "b2ab9cd9eca7f449e21de9e14d2979f7"
server_key_version: "encryptionkey@0"
I20260812 01:19:15.306203 30954 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260812 01:19:15.307018 30972 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:15.307216 30954 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:19:15.307328 30954 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786497534195564-24578-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786497534195564-24578-0/minicluster-data/ts-2/wal
uuid: "69c1f4ab561d4e84bb9ee86d605b1d2c"
format_stamp: "Formatted at 2026-08-12 01:19:15 on dist-test-slave-t3q3"
server_key: "1f5ebb90b9938bd92cec3a818f2cad7d"
server_key_iv: "b2ab9cd9eca7f449e21de9e14d2979f7"
server_key_version: "encryptionkey@0"
I20260812 01:19:15.307404 30954 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786497534195564-24578-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786497534195564-24578-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786497534195564-24578-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:19:15.324879 30954 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:19:15.327051 30954 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:19:15.327226 30954 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:19:15.327484 30954 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:19:15.327893 30954 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:19:15.327930 30954 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:15.327955 30954 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:19:15.327968 30954 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:15.334098 30954 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.131:44393
I20260812 01:19:15.334148 31151 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.131:44393 every 8 connection(s)
I20260812 01:19:15.334476 30954 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786497534195564-24578-0/minicluster-data/ts-2/data/info.pb
I20260812 01:19:15.338887 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 30954
I20260812 01:19:15.338997 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786497534195564-24578-0/minicluster-data/ts-2/wal/instance
I20260812 01:19:15.339175 24578 external_mini_cluster.cc:1428] Setting key 357491ba93b9a1f306c610aba5068757
Aug 12 01:19:15 dist-test-slave-t3q3 krb5kdc[30350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497555, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.131@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
I20260812 01:19:15.344659 30486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:15.335257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.131:37989 (local address 127.24.0.190:42255)
0812 01:19:15.335322 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:15.335324 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:15.335544 (+ 220us) server_negotiation.cc:407] Connection header received
0812 01:19:15.336627 (+ 1083us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:15.336628 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:15.336640 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:15.336697 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:15.337303 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:15.337804 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:15.338870 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:15.339210 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:15.339296 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:15.341776 (+ 2480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:15.341778 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:15.341778 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:15.341784 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:15.343302 (+ 1518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:15.343685 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:15.343686 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:15.343686 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:15.343739 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:15.343958 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:15.343959 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:15.343959 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:15.344157 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:15.344212 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:15.344524 (+ 312us) server_negotiation.cc:299] Negotiation successful
0812 01:19:15.344594 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 01:19:15.344933 31157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:15.335238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:42255 (local address 127.24.0.131:37989)
0812 01:19:15.335401 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:15.335440 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:15.336498 (+ 1058us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:15.336753 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:15.336757 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:15.337024 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:15.337220 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:15.337224 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:15.337924 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:15.337925 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:15.338770 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:15.338772 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:15.339330 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:15.339331 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:15.339483 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:15.340157 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:15.340167 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:15.341685 (+ 1518us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:19:15.343376 (+ 1691us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:15.343382 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:15.343384 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:15.343619 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:15.343798 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:15.343799 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:15.343799 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:15.343904 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:15.344221 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:19:15.344228 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:19:15.344400 (+ 172us) client_negotiation.cc:769] Sending connection context
0812 01:19:15.344499 (+ 99us) client_negotiation.cc:240] Negotiation successful
0812 01:19:15.344640 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":86,"threads_started":1}
I20260812 01:19:15.345809 31155 heartbeater.cc:347] Connected to a master server at 127.24.0.190:42255
I20260812 01:19:15.345942 31155 heartbeater.cc:464] Registering TS with master...
I20260812 01:19:15.346143 31155 heartbeater.cc:511] Master 127.24.0.190:42255 requested a full tablet report, sending...
I20260812 01:19:15.346606 30424 ts_manager.cc:194] Registered new tserver with Master: 69c1f4ab561d4e84bb9ee86d605b1d2c (127.24.0.131:44393)
I20260812 01:19:15.347074 30424 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.0.131@KRBTEST.COM'} at 127.24.0.131:37989
I20260812 01:19:15.351166 24578 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 01:19:15 dist-test-slave-t3q3 krb5kdc[30350](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497555, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 01:19:15 dist-test-slave-t3q3 krb5kdc[30350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497555, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.0.129@KRBTEST.COM
I20260812 01:19:15.376169 31165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:15.368021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37826 (local address 127.24.0.129:37117)
0812 01:19:15.368270 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:15.368276 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:15.368289 (+ 13us) server_negotiation.cc:407] Connection header received
0812 01:19:15.368327 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:15.368329 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:15.368345 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:15.368422 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:15.368981 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:15.369716 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:15.370472 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:15.370802 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:15.370862 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:15.373493 (+ 2631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:15.373502 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:15.373502 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:15.373509 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:15.375142 (+ 1633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:15.375439 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:15.375442 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:15.375442 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:15.375511 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:15.375661 (+ 150us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:15.375662 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:15.375662 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:15.375780 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:15.375885 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:15.375943 (+ 58us) server_negotiation.cc:299] Negotiation successful
0812 01:19:15.376004 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":209,"thread_start_us":173,"threads_started":1}
Aug 12 01:19:15 dist-test-slave-t3q3 krb5kdc[30350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497555, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
I20260812 01:19:15.384395 30486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:15.377496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41688 (local address 127.24.0.190:42255)
0812 01:19:15.377558 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:15.377560 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:15.377603 (+ 43us) server_negotiation.cc:407] Connection header received
0812 01:19:15.377684 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:15.377685 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:15.377696 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:15.377768 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:15.378245 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:15.378741 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:15.379460 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:15.379661 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:15.379746 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:15.381840 (+ 2094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:15.381842 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:15.381842 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:15.381850 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:15.383173 (+ 1323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:15.383517 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:15.383518 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:15.383518 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:15.383589 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:15.383804 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:15.383805 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:15.383805 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:15.383958 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:15.384067 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:15.384216 (+ 149us) server_negotiation.cc:299] Negotiation successful
0812 01:19:15.384294 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
W20260812 01:19:15.384771 30424 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41688
I20260812 01:19:15.385114 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 30519
I20260812 01:19:15.399107 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 30745
I20260812 01:19:15.407250 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 30954
I20260812 01:19:15.429206 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 30367
2026-08-12T01:19:15Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2205 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786497534195564-24578-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 01:19:15 dist-test-slave-t3q3 krb5kdc[31170](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 01:19:15 dist-test-slave-t3q3 krb5kdc[31170](info): set up 2 sockets
Aug 12 01:19:15 dist-test-slave-t3q3 krb5kdc[31170](info): commencing operation
krb5kdc: starting...
W20260812 01:19:16.697485 24578 mini_kdc.cc:121] Time spent starting KDC: real 1.241s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786497534195564-24578-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786497534195564-24578-0/krb5kdc/test-user.keytab.
Aug 12 01:19:16 dist-test-slave-t3q3 krb5kdc[31170](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497556, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T01:19:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:19:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.0.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.0.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.0.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab.
Entry for principal kudu/127.24.0.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab.
I20260812 01:19:16.823428 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786497534195564-24578-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786497534195564-24578-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786497534195564-24578-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.190:45405
--webserver_interface=127.24.0.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab
--principal=kudu/127.24.0.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:40581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.0.190:45405
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:19:16.907186 31189 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:19:16.907426 31189 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:19:16.907467 31189 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:19:16.908671 31189 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:19:16.908732 31189 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:19:16.908756 31189 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:19:16.908778 31189 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:19:16.908805 31189 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:19:16.910597 31189 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:40581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786497534195564-24578-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786497534195564-24578-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.0.190:45405
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab
--principal=kudu/127.24.0.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.190:45405
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.0.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31189
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786497534195564-24578-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:19:16.910892 31189 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:19:16.911125 31189 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:19:16.915870 31198 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:16.915871 31204 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:16.915931 31200 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:19:16.916249 31189 server_base.cc:1034] running on GCE node
I20260812 01:19:16.916532 31189 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:19:16.916899 31189 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:19:16.918069 31189 hybrid_clock.cc:648] HybridClock initialized: now 1786497556918045 us; error 43 us; skew 500 ppm
Aug 12 01:19:16 dist-test-slave-t3q3 krb5kdc[31170](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497556, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:19:16.920652 31189 init.cc:377] Logged in from keytab as kudu/127.24.0.190@KRBTEST.COM (short username kudu)
I20260812 01:19:16.920982 31189 fs_manager.cc:359] Metadata directory not provided
I20260812 01:19:16.921057 31189 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:19:16.921216 31189 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:19:16.922252 31189 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786497534195564-24578-0/minicluster-data/master-0/data/instance:
uuid: "9dbaddfc899c447ca2731952dd2b3ce9"
format_stamp: "Formatted at 2026-08-12 01:19:16 on dist-test-slave-t3q3"
server_key: "f266feac95267167ad13bb3cf8a73033"
server_key_iv: "533d6cb4ac442fd23f06728ba9dc9708"
server_key_version: "encryptionkey@0"
I20260812 01:19:16.922622 31189 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786497534195564-24578-0/minicluster-data/master-0/wal/instance:
uuid: "9dbaddfc899c447ca2731952dd2b3ce9"
format_stamp: "Formatted at 2026-08-12 01:19:16 on dist-test-slave-t3q3"
server_key: "f266feac95267167ad13bb3cf8a73033"
server_key_iv: "533d6cb4ac442fd23f06728ba9dc9708"
server_key_version: "encryptionkey@0"
I20260812 01:19:16.923928 31189 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 01:19:16.924759 31214 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:16.924962 31189 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:19:16.925091 31189 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786497534195564-24578-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786497534195564-24578-0/minicluster-data/master-0/wal
uuid: "9dbaddfc899c447ca2731952dd2b3ce9"
format_stamp: "Formatted at 2026-08-12 01:19:16 on dist-test-slave-t3q3"
server_key: "f266feac95267167ad13bb3cf8a73033"
server_key_iv: "533d6cb4ac442fd23f06728ba9dc9708"
server_key_version: "encryptionkey@0"
I20260812 01:19:16.925170 31189 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786497534195564-24578-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786497534195564-24578-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786497534195564-24578-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:19:16.941977 31189 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:19:16.943701 31189 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:19:16.943862 31189 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:19:16.947935 31189 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.190:45405
I20260812 01:19:16.947961 31310 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.190:45405 every 8 connection(s)
I20260812 01:19:16.948294 31189 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
I20260812 01:19:16.948846 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 31189
I20260812 01:19:16.948966 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786497534195564-24578-0/minicluster-data/master-0/wal/instance
I20260812 01:19:16.949149 24578 external_mini_cluster.cc:1428] Setting key d84cd486bf0c5b4d87399116d28d1a19
I20260812 01:19:16.949579 31312 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:19:16.952953 31312 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9dbaddfc899c447ca2731952dd2b3ce9: Bootstrap starting.
I20260812 01:19:16.953658 31312 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9dbaddfc899c447ca2731952dd2b3ce9: Neither blocks nor log segments found. Creating new log.
I20260812 01:19:16.954021 31312 log.cc:824] T 00000000000000000000000000000000 P 9dbaddfc899c447ca2731952dd2b3ce9: Log is configured to *not* fsync() on all Append() calls
I20260812 01:19:16.954787 31312 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9dbaddfc899c447ca2731952dd2b3ce9: No bootstrap required, opened a new log
Aug 12 01:19:16 dist-test-slave-t3q3 krb5kdc[31170](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497556, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
I20260812 01:19:16.957140 31312 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9dbaddfc899c447ca2731952dd2b3ce9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9dbaddfc899c447ca2731952dd2b3ce9" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 45405 } }
I20260812 01:19:16.957314 31312 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9dbaddfc899c447ca2731952dd2b3ce9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:19:16.957355 31312 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9dbaddfc899c447ca2731952dd2b3ce9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9dbaddfc899c447ca2731952dd2b3ce9, State: Initialized, Role: FOLLOWER
I20260812 01:19:16.957501 31312 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9dbaddfc899c447ca2731952dd2b3ce9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9dbaddfc899c447ca2731952dd2b3ce9" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 45405 } }
I20260812 01:19:16.957571 31312 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9dbaddfc899c447ca2731952dd2b3ce9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:19:16.957597 31312 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9dbaddfc899c447ca2731952dd2b3ce9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:19:16.957675 31312 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9dbaddfc899c447ca2731952dd2b3ce9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:19:16.958215 31312 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9dbaddfc899c447ca2731952dd2b3ce9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9dbaddfc899c447ca2731952dd2b3ce9" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 45405 } }
I20260812 01:19:16.958305 31312 leader_election.cc:302] T 00000000000000000000000000000000 P 9dbaddfc899c447ca2731952dd2b3ce9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9dbaddfc899c447ca2731952dd2b3ce9; no voters:
I20260812 01:19:16.958544 31312 leader_election.cc:288] T 00000000000000000000000000000000 P 9dbaddfc899c447ca2731952dd2b3ce9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 01:19:16.958581 31321 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9dbaddfc899c447ca2731952dd2b3ce9 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:19:16.958700 31321 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9dbaddfc899c447ca2731952dd2b3ce9 [term 1 LEADER]: Becoming Leader. State: Replica: 9dbaddfc899c447ca2731952dd2b3ce9, State: Running, Role: LEADER
I20260812 01:19:16.958841 31312 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9dbaddfc899c447ca2731952dd2b3ce9 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:19:16.958845 31321 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9dbaddfc899c447ca2731952dd2b3ce9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9dbaddfc899c447ca2731952dd2b3ce9" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 45405 } }
I20260812 01:19:16.959364 31322 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9dbaddfc899c447ca2731952dd2b3ce9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9dbaddfc899c447ca2731952dd2b3ce9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9dbaddfc899c447ca2731952dd2b3ce9" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 45405 } } }
I20260812 01:19:16.959458 31322 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9dbaddfc899c447ca2731952dd2b3ce9 [sys.catalog]: This master's current role is: LEADER
I20260812 01:19:16.959708 31323 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9dbaddfc899c447ca2731952dd2b3ce9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9dbaddfc899c447ca2731952dd2b3ce9. Latest consensus state: current_term: 1 leader_uuid: "9dbaddfc899c447ca2731952dd2b3ce9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9dbaddfc899c447ca2731952dd2b3ce9" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 45405 } } }
I20260812 01:19:16.960036 31323 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9dbaddfc899c447ca2731952dd2b3ce9 [sys.catalog]: This master's current role is: LEADER
I20260812 01:19:16.960249 31329 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:19:16.960276 31317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:16.949686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57908 (local address 127.24.0.190:45405)
0812 01:19:16.949877 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:16.949885 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:16.949910 (+ 25us) server_negotiation.cc:407] Connection header received
0812 01:19:16.950345 (+ 435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:16.950351 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:16.950394 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:16.950502 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:16.951285 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:16.952229 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:16.953275 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:16.953668 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:16.953806 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:16.956467 (+ 2661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:16.956471 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:16.956471 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:16.956481 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:16.958261 (+ 1780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:16.958711 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:16.958715 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:16.958716 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:16.958819 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:16.959016 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:16.959017 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:16.959017 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:16.959302 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:16.959435 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:16.959890 (+ 455us) server_negotiation.cc:299] Negotiation successful
0812 01:19:16.960040 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":101,"threads_started":1}
I20260812 01:19:16.960907 31329 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:19:16.962386 31329 catalog_manager.cc:1347] Generated new cluster ID: 6cd4954e461040bc9069eee3c00ccf70
I20260812 01:19:16.962445 31329 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 01:19:16.977919 31329 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:19:16.978647 31329 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:19:16.982648 31329 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9dbaddfc899c447ca2731952dd2b3ce9: Generated new TSK 0
I20260812 01:19:16.982873 31329 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.0.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.0.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.0.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.129.keytab.
Entry for principal kudu/127.24.0.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.129.keytab.
I20260812 01:19:17.015443 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786497534195564-24578-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786497534195564-24578-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786497534195564-24578-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786497534195564-24578-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.0.129:0
--local_ip_for_outbound_sockets=127.24.0.129
--webserver_interface=127.24.0.129
--webserver_port=0
--tserver_master_addrs=127.24.0.190:45405
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.129.keytab
--principal=kudu/127.24.0.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:40581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:19:17.124538 31353 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:19:17.124770 31353 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:19:17.124812 31353 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:19:17.125967 31353 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:19:17.126050 31353 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:19:17.126104 31353 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.0.129
I20260812 01:19:17.127619 31353 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:40581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786497534195564-24578-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786497534195564-24578-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.129.keytab
--principal=kudu/127.24.0.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.0.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786497534195564-24578-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.0.129
--webserver_port=0
--tserver_master_addrs=127.24.0.190:45405
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31353
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.0.129
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786497534195564-24578-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:19:17.127887 31353 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:19:17.128185 31353 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:19:17.133134 31364 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:17.133123 31366 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:17.133150 31363 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:19:17.133455 31353 server_base.cc:1034] running on GCE node
I20260812 01:19:17.133651 31353 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:19:17.133898 31353 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:19:17.135093 31353 hybrid_clock.cc:648] HybridClock initialized: now 1786497557135083 us; error 41 us; skew 500 ppm
Aug 12 01:19:17 dist-test-slave-t3q3 krb5kdc[31170](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497557, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:19:17.137773 31353 init.cc:377] Logged in from keytab as kudu/127.24.0.129@KRBTEST.COM (short username kudu)
I20260812 01:19:17.138204 31353 fs_manager.cc:359] Metadata directory not provided
I20260812 01:19:17.138280 31353 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:19:17.138430 31353 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:19:17.139521 31353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786497534195564-24578-0/minicluster-data/ts-0/data/instance:
uuid: "b678bad42abc4f408e56256d0d11a6f1"
format_stamp: "Formatted at 2026-08-12 01:19:17 on dist-test-slave-t3q3"
server_key: "4ca01ba8bbfe35b97d55f33e252952f0"
server_key_iv: "635ca9a115212ae5b6d5663bde0ff1d4"
server_key_version: "encryptionkey@0"
I20260812 01:19:17.139900 31353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786497534195564-24578-0/minicluster-data/ts-0/wal/instance:
uuid: "b678bad42abc4f408e56256d0d11a6f1"
format_stamp: "Formatted at 2026-08-12 01:19:17 on dist-test-slave-t3q3"
server_key: "4ca01ba8bbfe35b97d55f33e252952f0"
server_key_iv: "635ca9a115212ae5b6d5663bde0ff1d4"
server_key_version: "encryptionkey@0"
I20260812 01:19:17.141319 31353 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 01:19:17.142064 31374 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:17.142248 31353 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 01:19:17.142341 31353 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786497534195564-24578-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786497534195564-24578-0/minicluster-data/ts-0/wal
uuid: "b678bad42abc4f408e56256d0d11a6f1"
format_stamp: "Formatted at 2026-08-12 01:19:17 on dist-test-slave-t3q3"
server_key: "4ca01ba8bbfe35b97d55f33e252952f0"
server_key_iv: "635ca9a115212ae5b6d5663bde0ff1d4"
server_key_version: "encryptionkey@0"
I20260812 01:19:17.142421 31353 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786497534195564-24578-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786497534195564-24578-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786497534195564-24578-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:19:17.154088 31353 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:19:17.155987 31353 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:19:17.156188 31353 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:19:17.156440 31353 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:19:17.156775 31353 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:19:17.156832 31353 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:17.156877 31353 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:19:17.156904 31353 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:17.163379 31353 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.129:32837
I20260812 01:19:17.163416 31558 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.129:32837 every 8 connection(s)
I20260812 01:19:17.163713 31353 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786497534195564-24578-0/minicluster-data/ts-0/data/info.pb
Aug 12 01:19:17 dist-test-slave-t3q3 krb5kdc[31170](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497557, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.129@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
I20260812 01:19:17.172678 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 31353
I20260812 01:19:17.172788 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786497534195564-24578-0/minicluster-data/ts-0/wal/instance
I20260812 01:19:17.172804 31317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:17.164481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.129:44193 (local address 127.24.0.190:45405)
0812 01:19:17.164574 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:17.164577 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:17.164762 (+ 185us) server_negotiation.cc:407] Connection header received
0812 01:19:17.165230 (+ 468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:17.165231 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:17.165241 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:17.165294 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:17.166214 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:17.166713 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:17.167729 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:17.167918 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:17.167972 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:17.170121 (+ 2149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:17.170122 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:17.170123 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:17.170130 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:17.171412 (+ 1282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:17.171786 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:17.171787 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:17.171787 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:17.171841 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:17.172056 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:17.172115 (+ 59us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:17.172115 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:17.172237 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:17.172303 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:17.172586 (+ 283us) server_negotiation.cc:299] Negotiation successful
0812 01:19:17.172691 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 01:19:17.172869 31565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:17.164443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:45405 (local address 127.24.0.129:44193)
0812 01:19:17.164628 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:17.164665 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:17.165141 (+ 476us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:17.165348 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:17.165351 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:17.165602 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:17.166102 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:17.166117 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:17.166830 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:17.166831 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:17.167645 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:17.167648 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:17.167993 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:17.167993 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:17.168122 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:17.168661 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:17.168670 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:17.170033 (+ 1363us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:19:17.171487 (+ 1454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:17.171494 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:17.171496 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:17.171714 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:17.171901 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:17.171901 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:17.171902 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:17.172000 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:17.172308 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:19:17.172313 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:19:17.172429 (+ 116us) client_negotiation.cc:769] Sending connection context
0812 01:19:17.172527 (+ 98us) client_negotiation.cc:240] Negotiation successful
0812 01:19:17.172645 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":106,"threads_started":1}
I20260812 01:19:17.173005 24578 external_mini_cluster.cc:1428] Setting key 668a318291d41f93577fd9140f0378da
I20260812 01:19:17.173465 31559 heartbeater.cc:347] Connected to a master server at 127.24.0.190:45405
I20260812 01:19:17.173568 31559 heartbeater.cc:464] Registering TS with master...
I20260812 01:19:17.173727 31559 heartbeater.cc:511] Master 127.24.0.190:45405 requested a full tablet report, sending...
I20260812 01:19:17.174341 31250 ts_manager.cc:194] Registered new tserver with Master: b678bad42abc4f408e56256d0d11a6f1 (127.24.0.129:32837)
I20260812 01:19:17.174952 31250 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.0.129@KRBTEST.COM'} at 127.24.0.129:44193
WARNING: no policy specified for kudu/127.24.0.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.0.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.0.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.130.keytab.
Entry for principal kudu/127.24.0.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.130.keytab.
I20260812 01:19:17.200578 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786497534195564-24578-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786497534195564-24578-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786497534195564-24578-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786497534195564-24578-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.0.130:0
--local_ip_for_outbound_sockets=127.24.0.130
--webserver_interface=127.24.0.130
--webserver_port=0
--tserver_master_addrs=127.24.0.190:45405
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.130.keytab
--principal=kudu/127.24.0.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:40581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:19:17.284587 31568 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:19:17.284806 31568 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:19:17.284832 31568 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:19:17.286052 31568 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:19:17.286096 31568 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:19:17.286123 31568 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.0.130
I20260812 01:19:17.287597 31568 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:40581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786497534195564-24578-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786497534195564-24578-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.130.keytab
--principal=kudu/127.24.0.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.0.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786497534195564-24578-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.0.130
--webserver_port=0
--tserver_master_addrs=127.24.0.190:45405
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31568
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.0.130
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786497534195564-24578-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:19:17.287801 31568 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:19:17.287978 31568 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:19:17.292963 31578 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:17.293017 31576 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:17.292945 31580 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:19:17.293263 31568 server_base.cc:1034] running on GCE node
I20260812 01:19:17.293495 31568 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:19:17.293742 31568 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:19:17.294911 31568 hybrid_clock.cc:648] HybridClock initialized: now 1786497557294891 us; error 29 us; skew 500 ppm
Aug 12 01:19:17 dist-test-slave-t3q3 krb5kdc[31170](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497557, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:19:17.297467 31568 init.cc:377] Logged in from keytab as kudu/127.24.0.130@KRBTEST.COM (short username kudu)
I20260812 01:19:17.297875 31568 fs_manager.cc:359] Metadata directory not provided
I20260812 01:19:17.297948 31568 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:19:17.298110 31568 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:19:17.299199 31568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786497534195564-24578-0/minicluster-data/ts-1/data/instance:
uuid: "81e26f32389c48e2921a421f51878505"
format_stamp: "Formatted at 2026-08-12 01:19:17 on dist-test-slave-t3q3"
server_key: "05bed740f1e87889530a9135eadb8e5a"
server_key_iv: "cd680c35edf959bf739d576eed9e8343"
server_key_version: "encryptionkey@0"
I20260812 01:19:17.299585 31568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786497534195564-24578-0/minicluster-data/ts-1/wal/instance:
uuid: "81e26f32389c48e2921a421f51878505"
format_stamp: "Formatted at 2026-08-12 01:19:17 on dist-test-slave-t3q3"
server_key: "05bed740f1e87889530a9135eadb8e5a"
server_key_iv: "cd680c35edf959bf739d576eed9e8343"
server_key_version: "encryptionkey@0"
I20260812 01:19:17.301059 31568 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 01:19:17.301823 31589 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:17.301996 31568 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 01:19:17.302122 31568 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786497534195564-24578-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786497534195564-24578-0/minicluster-data/ts-1/wal
uuid: "81e26f32389c48e2921a421f51878505"
format_stamp: "Formatted at 2026-08-12 01:19:17 on dist-test-slave-t3q3"
server_key: "05bed740f1e87889530a9135eadb8e5a"
server_key_iv: "cd680c35edf959bf739d576eed9e8343"
server_key_version: "encryptionkey@0"
I20260812 01:19:17.302202 31568 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786497534195564-24578-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786497534195564-24578-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786497534195564-24578-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:19:17.323559 31568 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:19:17.325663 31568 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:19:17.325832 31568 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:19:17.326090 31568 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:19:17.326416 31568 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:19:17.326471 31568 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:17.326515 31568 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:19:17.326540 31568 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:17.332906 31568 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.130:39589
I20260812 01:19:17.332948 31767 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.130:39589 every 8 connection(s)
I20260812 01:19:17.333250 31568 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786497534195564-24578-0/minicluster-data/ts-1/data/info.pb
I20260812 01:19:17.336285 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 31568
I20260812 01:19:17.336400 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786497534195564-24578-0/minicluster-data/ts-1/wal/instance
I20260812 01:19:17.336587 24578 external_mini_cluster.cc:1428] Setting key 2f94fd6adbc252a37920bb1fc0f1a470
Aug 12 01:19:17 dist-test-slave-t3q3 krb5kdc[31170](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497557, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.130@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
I20260812 01:19:17.343680 31317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:17.333973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.130:36203 (local address 127.24.0.190:45405)
0812 01:19:17.334045 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:17.334047 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:17.334256 (+ 209us) server_negotiation.cc:407] Connection header received
0812 01:19:17.334813 (+ 557us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:17.334815 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:17.334829 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:17.334885 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:17.335871 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:17.336649 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:17.337865 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:17.338197 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:17.338269 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:17.340815 (+ 2546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:17.340816 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:17.340816 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:17.340825 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:17.342238 (+ 1413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:17.342650 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:17.342651 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:17.342652 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:17.342721 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:17.342999 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:17.343000 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:17.343001 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:17.343121 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:17.343201 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:17.343484 (+ 283us) server_negotiation.cc:299] Negotiation successful
0812 01:19:17.343591 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260812 01:19:17.343852 31771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:17.333945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:45405 (local address 127.24.0.130:36203)
0812 01:19:17.334112 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:17.334159 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:17.334678 (+ 519us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:17.334957 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:17.334961 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:17.335241 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:17.335756 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:17.335761 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:17.336788 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:17.336790 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:17.337768 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:17.337771 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:17.338288 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:17.338288 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:17.338392 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:17.338924 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:17.338932 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:17.340709 (+ 1777us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:19:17.342332 (+ 1623us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:17.342339 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:17.342341 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:17.342569 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:17.342791 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:17.342792 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:17.342792 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:17.342919 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:17.343192 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:19:17.343198 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:19:17.343343 (+ 145us) client_negotiation.cc:769] Sending connection context
0812 01:19:17.343450 (+ 107us) client_negotiation.cc:240] Negotiation successful
0812 01:19:17.343610 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":71,"threads_started":1}
I20260812 01:19:17.344408 31768 heartbeater.cc:347] Connected to a master server at 127.24.0.190:45405
I20260812 01:19:17.344520 31768 heartbeater.cc:464] Registering TS with master...
I20260812 01:19:17.344672 31768 heartbeater.cc:511] Master 127.24.0.190:45405 requested a full tablet report, sending...
I20260812 01:19:17.345101 31250 ts_manager.cc:194] Registered new tserver with Master: 81e26f32389c48e2921a421f51878505 (127.24.0.130:39589)
I20260812 01:19:17.345502 31250 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.0.130@KRBTEST.COM'} at 127.24.0.130:36203
WARNING: no policy specified for kudu/127.24.0.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.0.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.0.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.131.keytab.
Entry for principal kudu/127.24.0.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.131.keytab.
I20260812 01:19:17.363169 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786497534195564-24578-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786497534195564-24578-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786497534195564-24578-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786497534195564-24578-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.0.131:0
--local_ip_for_outbound_sockets=127.24.0.131
--webserver_interface=127.24.0.131
--webserver_port=0
--tserver_master_addrs=127.24.0.190:45405
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.131.keytab
--principal=kudu/127.24.0.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:40581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:19:17.447227 31776 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:19:17.447438 31776 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:19:17.447461 31776 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:19:17.448664 31776 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:19:17.448709 31776 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:19:17.448735 31776 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.0.131
I20260812 01:19:17.450286 31776 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:40581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786497534195564-24578-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786497534195564-24578-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.131.keytab
--principal=kudu/127.24.0.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.0.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786497534195564-24578-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.0.131
--webserver_port=0
--tserver_master_addrs=127.24.0.190:45405
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31776
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.0.131
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786497534195564-24578-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:19:17.450519 31776 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:19:17.450786 31776 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:19:17.455907 31790 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:19:17.456179 31776 server_base.cc:1034] running on GCE node
W20260812 01:19:17.455897 31783 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:17.455922 31787 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:19:17.456521 31776 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:19:17.456800 31776 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:19:17.457993 31776 hybrid_clock.cc:648] HybridClock initialized: now 1786497557457971 us; error 43 us; skew 500 ppm
Aug 12 01:19:17 dist-test-slave-t3q3 krb5kdc[31170](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497557, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:19:17.461138 31776 init.cc:377] Logged in from keytab as kudu/127.24.0.131@KRBTEST.COM (short username kudu)
I20260812 01:19:17.461526 31776 fs_manager.cc:359] Metadata directory not provided
I20260812 01:19:17.461611 31776 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:19:17.461766 31776 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:19:17.462826 31776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786497534195564-24578-0/minicluster-data/ts-2/data/instance:
uuid: "e9b55f3a33444cebad4bf814a5599cb6"
format_stamp: "Formatted at 2026-08-12 01:19:17 on dist-test-slave-t3q3"
server_key: "db3ed034eab5c69decb3dc2958b33b64"
server_key_iv: "02a9ff5212f0b489435c5df86e6b2b16"
server_key_version: "encryptionkey@0"
I20260812 01:19:17.463193 31776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786497534195564-24578-0/minicluster-data/ts-2/wal/instance:
uuid: "e9b55f3a33444cebad4bf814a5599cb6"
format_stamp: "Formatted at 2026-08-12 01:19:17 on dist-test-slave-t3q3"
server_key: "db3ed034eab5c69decb3dc2958b33b64"
server_key_iv: "02a9ff5212f0b489435c5df86e6b2b16"
server_key_version: "encryptionkey@0"
I20260812 01:19:17.464620 31776 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 01:19:17.465396 31799 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:17.465600 31776 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 01:19:17.465729 31776 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786497534195564-24578-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786497534195564-24578-0/minicluster-data/ts-2/wal
uuid: "e9b55f3a33444cebad4bf814a5599cb6"
format_stamp: "Formatted at 2026-08-12 01:19:17 on dist-test-slave-t3q3"
server_key: "db3ed034eab5c69decb3dc2958b33b64"
server_key_iv: "02a9ff5212f0b489435c5df86e6b2b16"
server_key_version: "encryptionkey@0"
I20260812 01:19:17.465808 31776 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786497534195564-24578-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786497534195564-24578-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786497534195564-24578-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:19:17.481863 31776 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:19:17.484508 31776 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:19:17.484679 31776 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:19:17.484929 31776 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:19:17.485260 31776 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:19:17.485316 31776 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:17.485352 31776 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:19:17.485399 31776 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:17.491655 31776 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.131:37267
I20260812 01:19:17.491693 31974 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.131:37267 every 8 connection(s)
I20260812 01:19:17.492020 31776 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786497534195564-24578-0/minicluster-data/ts-2/data/info.pb
Aug 12 01:19:17 dist-test-slave-t3q3 krb5kdc[31170](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497557, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.131@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
I20260812 01:19:17.498776 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 31776
I20260812 01:19:17.498878 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786497534195564-24578-0/minicluster-data/ts-2/wal/instance
I20260812 01:19:17.499082 24578 external_mini_cluster.cc:1428] Setting key f114fa1ec09fecb7c699f6037299114e
I20260812 01:19:17.501534 31317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:17.492798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.131:36249 (local address 127.24.0.190:45405)
0812 01:19:17.492879 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:17.492882 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:17.493223 (+ 341us) server_negotiation.cc:407] Connection header received
0812 01:19:17.493688 (+ 465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:17.493689 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:17.493700 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:17.493750 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:17.494656 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:17.495172 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:17.496185 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:17.496399 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:17.496465 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:17.498515 (+ 2050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:17.498516 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:17.498516 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:17.498523 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:17.500257 (+ 1734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:17.500633 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:17.500634 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:17.500634 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:17.500699 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:17.500925 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:17.500926 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:17.500927 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:17.501072 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:17.501135 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:17.501372 (+ 237us) server_negotiation.cc:299] Negotiation successful
0812 01:19:17.501459 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 01:19:17.501648 31978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:17.492787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:45405 (local address 127.24.0.131:36249)
0812 01:19:17.492976 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:17.493056 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:17.493604 (+ 548us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:17.493810 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:17.493814 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:17.494068 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:17.494561 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:17.494566 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:17.495255 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:17.495255 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:17.496051 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:17.496053 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:17.496483 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:17.496484 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:17.496603 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:17.497048 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:17.497056 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:17.498425 (+ 1369us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:19:17.500323 (+ 1898us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:17.500329 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:17.500331 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:17.500544 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:17.500777 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:17.500777 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:17.500778 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:17.500862 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:17.501138 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:19:17.501143 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:19:17.501254 (+ 111us) client_negotiation.cc:769] Sending connection context
0812 01:19:17.501336 (+ 82us) client_negotiation.cc:240] Negotiation successful
0812 01:19:17.501452 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":101,"threads_started":1}
I20260812 01:19:17.502174 31975 heartbeater.cc:347] Connected to a master server at 127.24.0.190:45405
I20260812 01:19:17.502286 31975 heartbeater.cc:464] Registering TS with master...
I20260812 01:19:17.502436 31975 heartbeater.cc:511] Master 127.24.0.190:45405 requested a full tablet report, sending...
I20260812 01:19:17.502836 31250 ts_manager.cc:194] Registered new tserver with Master: e9b55f3a33444cebad4bf814a5599cb6 (127.24.0.131:37267)
I20260812 01:19:17.503233 31250 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.0.131@KRBTEST.COM'} at 127.24.0.131:36249
I20260812 01:19:17.510948 24578 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 01:19:17.518128 31317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:17.512286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57922 (local address 127.24.0.190:45405)
0812 01:19:17.512354 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:17.512356 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:17.512455 (+ 99us) server_negotiation.cc:407] Connection header received
0812 01:19:17.512528 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:17.512529 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:17.512538 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:17.512577 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:17.513208 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:17.513668 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:17.514369 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:17.514579 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:17.514645 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:17.515789 (+ 1144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:17.515790 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:17.515790 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:17.515796 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:17.517108 (+ 1312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:17.517415 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:17.517415 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:17.517416 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:17.517467 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:17.517645 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:17.517645 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:17.517646 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:17.517784 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:17.517868 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:17.517947 (+ 79us) server_negotiation.cc:299] Negotiation successful
0812 01:19:17.518010 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 01:19:17.519832 31250 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57922:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 01:19:17.521611 31250 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 01:19:17.528771 31997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:17.524016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.131:37267 (local address 127.0.0.1:60354)
0812 01:19:17.524250 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:17.524297 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:17.524372 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:17.524976 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:17.524980 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:17.524982 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:19:17.525054 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:17.525056 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:17.526185 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:17.526186 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:17.527389 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:17.527391 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:17.528477 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:17.528478 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:17.528555 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:17.528569 (+ 14us) client_negotiation.cc:769] Sending connection context
0812 01:19:17.528639 (+ 70us) client_negotiation.cc:240] Negotiation successful
0812 01:19:17.528691 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":87,"threads_started":1}
I20260812 01:19:17.528934 32005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:17.524606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37652 (local address 127.24.0.130:39589)
0812 01:19:17.524859 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:17.524863 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:17.524881 (+ 18us) server_negotiation.cc:407] Connection header received
0812 01:19:17.524917 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:17.524919 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:17.524954 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:17.525039 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:19:17.525492 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:17.526614 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:17.527893 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:17.528534 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:17.528635 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:17.528695 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:17.528810 (+ 115us) server_negotiation.cc:299] Negotiation successful
0812 01:19:17.528853 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":211,"thread_start_us":90,"threads_started":1}
I20260812 01:19:17.529211 32003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:17.524367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60354 (local address 127.24.0.131:37267)
0812 01:19:17.524794 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:17.524798 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:17.524807 (+ 9us) server_negotiation.cc:407] Connection header received
0812 01:19:17.524866 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:17.524869 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:17.524886 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:17.524973 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:19:17.525155 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:17.526083 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:17.527497 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:17.528290 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:17.528381 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:17.528441 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:17.528951 (+ 510us) server_negotiation.cc:299] Negotiation successful
0812 01:19:17.528993 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":397,"thread_start_us":111,"threads_started":1}
I20260812 01:19:17.529227 32000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:17.524344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42808 (local address 127.24.0.129:32837)
0812 01:19:17.524509 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:17.524514 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:17.525170 (+ 656us) server_negotiation.cc:407] Connection header received
0812 01:19:17.525316 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:17.525319 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:17.525335 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:17.525409 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:19:17.525587 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:17.526644 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:17.527970 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:17.528780 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:17.528826 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:17.528886 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:17.529108 (+ 222us) server_negotiation.cc:299] Negotiation successful
0812 01:19:17.529151 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":98,"threads_started":1}
I20260812 01:19:17.529064 31998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:17.524036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.130:39589 (local address 127.0.0.1:37652)
0812 01:19:17.524491 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:17.524501 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:17.524555 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:17.525136 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:17.525137 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:17.525139 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:19:17.525200 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:17.525201 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:17.526730 (+ 1529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:17.526731 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:17.527804 (+ 1073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:17.527807 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:17.528638 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:17.528639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:17.528711 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:17.528716 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 01:19:17.528776 (+ 60us) client_negotiation.cc:240] Negotiation successful
0812 01:19:17.528821 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":437,"mutex_wait_us":9,"thread_start_us":154,"threads_started":1}
I20260812 01:19:17.529441 31996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:17.524039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.129:32837 (local address 127.0.0.1:42808)
0812 01:19:17.525091 (+ 1052us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:17.525102 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:17.525204 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:17.525435 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:17.525436 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:17.525439 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:19:17.525492 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:17.525493 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:17.526767 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:17.526768 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:17.527866 (+ 1098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:17.527869 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:17.528952 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:17.528953 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:17.529009 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:17.529013 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 01:19:17.529308 (+ 295us) client_negotiation.cc:240] Negotiation successful
0812 01:19:17.529361 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1037,"mutex_wait_us":2,"thread_start_us":143,"threads_started":1}
I20260812 01:19:17.530117 31663 tablet_service.cc:1467] Processing CreateTablet for tablet 4a89adf633bb4853a2572ca1b06bc615 (DEFAULT_TABLE table=test-table [id=8c67db97cb7348379bc0ca1dbe5f8bcf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:19:17.530125 31873 tablet_service.cc:1467] Processing CreateTablet for tablet 4a89adf633bb4853a2572ca1b06bc615 (DEFAULT_TABLE table=test-table [id=8c67db97cb7348379bc0ca1dbe5f8bcf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:19:17.530191 31456 tablet_service.cc:1467] Processing CreateTablet for tablet 4a89adf633bb4853a2572ca1b06bc615 (DEFAULT_TABLE table=test-table [id=8c67db97cb7348379bc0ca1dbe5f8bcf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:19:17.530490 31663 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a89adf633bb4853a2572ca1b06bc615. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:19:17.530490 31873 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a89adf633bb4853a2572ca1b06bc615. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:19:17.530565 31456 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a89adf633bb4853a2572ca1b06bc615. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:19:17.533404 32007 tablet_bootstrap.cc:492] T 4a89adf633bb4853a2572ca1b06bc615 P b678bad42abc4f408e56256d0d11a6f1: Bootstrap starting.
I20260812 01:19:17.534029 32010 tablet_bootstrap.cc:492] T 4a89adf633bb4853a2572ca1b06bc615 P 81e26f32389c48e2921a421f51878505: Bootstrap starting.
I20260812 01:19:17.534173 32007 tablet_bootstrap.cc:654] T 4a89adf633bb4853a2572ca1b06bc615 P b678bad42abc4f408e56256d0d11a6f1: Neither blocks nor log segments found. Creating new log.
I20260812 01:19:17.534785 32010 tablet_bootstrap.cc:654] T 4a89adf633bb4853a2572ca1b06bc615 P 81e26f32389c48e2921a421f51878505: Neither blocks nor log segments found. Creating new log.
I20260812 01:19:17.534850 32007 log.cc:824] T 4a89adf633bb4853a2572ca1b06bc615 P b678bad42abc4f408e56256d0d11a6f1: Log is configured to *not* fsync() on all Append() calls
I20260812 01:19:17.535419 32010 log.cc:824] T 4a89adf633bb4853a2572ca1b06bc615 P 81e26f32389c48e2921a421f51878505: Log is configured to *not* fsync() on all Append() calls
I20260812 01:19:17.535823 32011 tablet_bootstrap.cc:492] T 4a89adf633bb4853a2572ca1b06bc615 P e9b55f3a33444cebad4bf814a5599cb6: Bootstrap starting.
I20260812 01:19:17.536092 32007 tablet_bootstrap.cc:492] T 4a89adf633bb4853a2572ca1b06bc615 P b678bad42abc4f408e56256d0d11a6f1: No bootstrap required, opened a new log
I20260812 01:19:17.536185 32007 ts_tablet_manager.cc:1397] T 4a89adf633bb4853a2572ca1b06bc615 P b678bad42abc4f408e56256d0d11a6f1: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 01:19:17.536305 32010 tablet_bootstrap.cc:492] T 4a89adf633bb4853a2572ca1b06bc615 P 81e26f32389c48e2921a421f51878505: No bootstrap required, opened a new log
I20260812 01:19:17.536396 32010 ts_tablet_manager.cc:1397] T 4a89adf633bb4853a2572ca1b06bc615 P 81e26f32389c48e2921a421f51878505: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 01:19:17.536578 32011 tablet_bootstrap.cc:654] T 4a89adf633bb4853a2572ca1b06bc615 P e9b55f3a33444cebad4bf814a5599cb6: Neither blocks nor log segments found. Creating new log.
I20260812 01:19:17.537207 32011 log.cc:824] T 4a89adf633bb4853a2572ca1b06bc615 P e9b55f3a33444cebad4bf814a5599cb6: Log is configured to *not* fsync() on all Append() calls
I20260812 01:19:17.538054 32011 tablet_bootstrap.cc:492] T 4a89adf633bb4853a2572ca1b06bc615 P e9b55f3a33444cebad4bf814a5599cb6: No bootstrap required, opened a new log
I20260812 01:19:17.538136 32011 ts_tablet_manager.cc:1397] T 4a89adf633bb4853a2572ca1b06bc615 P e9b55f3a33444cebad4bf814a5599cb6: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 01:19:17.538091 32010 raft_consensus.cc:348] T 4a89adf633bb4853a2572ca1b06bc615 P 81e26f32389c48e2921a421f51878505 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9b55f3a33444cebad4bf814a5599cb6" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 37267 } } peers { permanent_uuid: "81e26f32389c48e2921a421f51878505" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39589 } } peers { permanent_uuid: "b678bad42abc4f408e56256d0d11a6f1" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 32837 } }
I20260812 01:19:17.538322 32010 raft_consensus.cc:374] T 4a89adf633bb4853a2572ca1b06bc615 P 81e26f32389c48e2921a421f51878505 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:19:17.538391 32010 raft_consensus.cc:729] T 4a89adf633bb4853a2572ca1b06bc615 P 81e26f32389c48e2921a421f51878505 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81e26f32389c48e2921a421f51878505, State: Initialized, Role: FOLLOWER
I20260812 01:19:17.538528 32010 consensus_queue.cc:260] T 4a89adf633bb4853a2572ca1b06bc615 P 81e26f32389c48e2921a421f51878505 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9b55f3a33444cebad4bf814a5599cb6" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 37267 } } peers { permanent_uuid: "81e26f32389c48e2921a421f51878505" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39589 } } peers { permanent_uuid: "b678bad42abc4f408e56256d0d11a6f1" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 32837 } }
I20260812 01:19:17.538558 32007 raft_consensus.cc:348] T 4a89adf633bb4853a2572ca1b06bc615 P b678bad42abc4f408e56256d0d11a6f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9b55f3a33444cebad4bf814a5599cb6" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 37267 } } peers { permanent_uuid: "81e26f32389c48e2921a421f51878505" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39589 } } peers { permanent_uuid: "b678bad42abc4f408e56256d0d11a6f1" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 32837 } }
I20260812 01:19:17.538775 32007 raft_consensus.cc:374] T 4a89adf633bb4853a2572ca1b06bc615 P b678bad42abc4f408e56256d0d11a6f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:19:17.538829 32007 raft_consensus.cc:729] T 4a89adf633bb4853a2572ca1b06bc615 P b678bad42abc4f408e56256d0d11a6f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b678bad42abc4f408e56256d0d11a6f1, State: Initialized, Role: FOLLOWER
I20260812 01:19:17.538837 32010 ts_tablet_manager.cc:1428] T 4a89adf633bb4853a2572ca1b06bc615 P 81e26f32389c48e2921a421f51878505: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 01:19:17.538951 31768 heartbeater.cc:503] Master 127.24.0.190:45405 was elected leader, sending a full tablet report...
I20260812 01:19:17.538978 32007 consensus_queue.cc:260] T 4a89adf633bb4853a2572ca1b06bc615 P b678bad42abc4f408e56256d0d11a6f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9b55f3a33444cebad4bf814a5599cb6" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 37267 } } peers { permanent_uuid: "81e26f32389c48e2921a421f51878505" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39589 } } peers { permanent_uuid: "b678bad42abc4f408e56256d0d11a6f1" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 32837 } }
I20260812 01:19:17.539335 31559 heartbeater.cc:503] Master 127.24.0.190:45405 was elected leader, sending a full tablet report...
I20260812 01:19:17.539664 32007 ts_tablet_manager.cc:1428] T 4a89adf633bb4853a2572ca1b06bc615 P b678bad42abc4f408e56256d0d11a6f1: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 01:19:17.540387 32011 raft_consensus.cc:348] T 4a89adf633bb4853a2572ca1b06bc615 P e9b55f3a33444cebad4bf814a5599cb6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9b55f3a33444cebad4bf814a5599cb6" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 37267 } } peers { permanent_uuid: "81e26f32389c48e2921a421f51878505" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39589 } } peers { permanent_uuid: "b678bad42abc4f408e56256d0d11a6f1" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 32837 } }
I20260812 01:19:17.540555 32011 raft_consensus.cc:374] T 4a89adf633bb4853a2572ca1b06bc615 P e9b55f3a33444cebad4bf814a5599cb6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:19:17.540643 32011 raft_consensus.cc:729] T 4a89adf633bb4853a2572ca1b06bc615 P e9b55f3a33444cebad4bf814a5599cb6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9b55f3a33444cebad4bf814a5599cb6, State: Initialized, Role: FOLLOWER
I20260812 01:19:17.540792 32011 consensus_queue.cc:260] T 4a89adf633bb4853a2572ca1b06bc615 P e9b55f3a33444cebad4bf814a5599cb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9b55f3a33444cebad4bf814a5599cb6" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 37267 } } peers { permanent_uuid: "81e26f32389c48e2921a421f51878505" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39589 } } peers { permanent_uuid: "b678bad42abc4f408e56256d0d11a6f1" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 32837 } }
I20260812 01:19:17.541100 32011 ts_tablet_manager.cc:1428] T 4a89adf633bb4853a2572ca1b06bc615 P e9b55f3a33444cebad4bf814a5599cb6: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 01:19:17.541109 31975 heartbeater.cc:503] Master 127.24.0.190:45405 was elected leader, sending a full tablet report...
W20260812 01:19:17.583784 31769 tablet.cc:2369] T 4a89adf633bb4853a2572ca1b06bc615 P 81e26f32389c48e2921a421f51878505: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 01:19:17.664517 31563 tablet.cc:2369] T 4a89adf633bb4853a2572ca1b06bc615 P b678bad42abc4f408e56256d0d11a6f1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 01:19:17.742640 31977 tablet.cc:2369] T 4a89adf633bb4853a2572ca1b06bc615 P e9b55f3a33444cebad4bf814a5599cb6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 01:19:17.772094 32025 raft_consensus.cc:482] T 4a89adf633bb4853a2572ca1b06bc615 P b678bad42abc4f408e56256d0d11a6f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 01:19:17.772257 32025 raft_consensus.cc:504] T 4a89adf633bb4853a2572ca1b06bc615 P b678bad42abc4f408e56256d0d11a6f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9b55f3a33444cebad4bf814a5599cb6" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 37267 } } peers { permanent_uuid: "81e26f32389c48e2921a421f51878505" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39589 } } peers { permanent_uuid: "b678bad42abc4f408e56256d0d11a6f1" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 32837 } }
I20260812 01:19:17.772702 32025 leader_election.cc:288] T 4a89adf633bb4853a2572ca1b06bc615 P b678bad42abc4f408e56256d0d11a6f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9b55f3a33444cebad4bf814a5599cb6 (127.24.0.131:37267), 81e26f32389c48e2921a421f51878505 (127.24.0.130:39589)
I20260812 01:19:17.775715 32005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:17.773058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.129:42519 (local address 127.24.0.130:39589)
0812 01:19:17.773125 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:17.773128 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:17.773162 (+ 34us) server_negotiation.cc:407] Connection header received
0812 01:19:17.773263 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:17.773264 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:17.773280 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:17.773356 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:19:17.773498 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:17.773969 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:17.774947 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:17.775352 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:17.775444 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:17.775453 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:17.775606 (+ 153us) server_negotiation.cc:299] Negotiation successful
0812 01:19:17.775646 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 01:19:17.775766 32027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:17.772925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.130:39589 (local address 127.24.0.129:42519)
0812 01:19:17.773097 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:17.773110 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:17.773181 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:17.773363 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:17.773364 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:17.773367 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:19:17.773430 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:17.773431 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:17.774057 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:17.774058 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:17.774851 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:17.774853 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:17.775446 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:17.775447 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:17.775514 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:17.775518 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 01:19:17.775597 (+ 79us) client_negotiation.cc:240] Negotiation successful
0812 01:19:17.775640 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":106,"threads_started":1}
I20260812 01:19:17.775763 32003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:17.773038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.129:38385 (local address 127.24.0.131:37267)
0812 01:19:17.773090 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:17.773094 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:17.773260 (+ 166us) server_negotiation.cc:407] Connection header received
0812 01:19:17.773336 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:17.773337 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:17.773349 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:17.773426 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:19:17.773606 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:17.774053 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:17.775008 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:17.775398 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:17.775487 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:17.775496 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:17.775654 (+ 158us) server_negotiation.cc:299] Negotiation successful
0812 01:19:17.775691 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 01:19:17.775784 32028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:17.772925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.131:37267 (local address 127.24.0.129:38385)
0812 01:19:17.773159 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:17.773169 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:17.773271 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:17.773431 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:17.773432 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:17.773434 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:19:17.773513 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:17.773515 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:17.774140 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:17.774142 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:17.774906 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:17.774907 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:17.775488 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:17.775489 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:17.775570 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:17.775574 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 01:19:17.775646 (+ 72us) client_negotiation.cc:240] Negotiation successful
0812 01:19:17.775692 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221,"mutex_wait_us":2,"thread_start_us":103,"threads_started":1}
I20260812 01:19:17.776134 31695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a89adf633bb4853a2572ca1b06bc615" candidate_uuid: "b678bad42abc4f408e56256d0d11a6f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81e26f32389c48e2921a421f51878505" is_pre_election: true
I20260812 01:19:17.776309 31695 raft_consensus.cc:2454] T 4a89adf633bb4853a2572ca1b06bc615 P 81e26f32389c48e2921a421f51878505 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b678bad42abc4f408e56256d0d11a6f1 in term 0.
I20260812 01:19:17.776252 31908 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a89adf633bb4853a2572ca1b06bc615" candidate_uuid: "b678bad42abc4f408e56256d0d11a6f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9b55f3a33444cebad4bf814a5599cb6" is_pre_election: true
I20260812 01:19:17.776438 31908 raft_consensus.cc:2454] T 4a89adf633bb4853a2572ca1b06bc615 P e9b55f3a33444cebad4bf814a5599cb6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b678bad42abc4f408e56256d0d11a6f1 in term 0.
I20260812 01:19:17.776561 31379 leader_election.cc:302] T 4a89adf633bb4853a2572ca1b06bc615 P b678bad42abc4f408e56256d0d11a6f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 81e26f32389c48e2921a421f51878505, b678bad42abc4f408e56256d0d11a6f1; no voters:
I20260812 01:19:17.776710 32025 raft_consensus.cc:2781] T 4a89adf633bb4853a2572ca1b06bc615 P b678bad42abc4f408e56256d0d11a6f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 01:19:17.776772 32025 raft_consensus.cc:482] T 4a89adf633bb4853a2572ca1b06bc615 P b678bad42abc4f408e56256d0d11a6f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 01:19:17.776794 32025 raft_consensus.cc:3037] T 4a89adf633bb4853a2572ca1b06bc615 P b678bad42abc4f408e56256d0d11a6f1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:19:17.777529 32025 raft_consensus.cc:504] T 4a89adf633bb4853a2572ca1b06bc615 P b678bad42abc4f408e56256d0d11a6f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9b55f3a33444cebad4bf814a5599cb6" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 37267 } } peers { permanent_uuid: "81e26f32389c48e2921a421f51878505" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39589 } } peers { permanent_uuid: "b678bad42abc4f408e56256d0d11a6f1" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 32837 } }
I20260812 01:19:17.777689 32025 leader_election.cc:288] T 4a89adf633bb4853a2572ca1b06bc615 P b678bad42abc4f408e56256d0d11a6f1 [CANDIDATE]: Term 1 election: Requested vote from peers e9b55f3a33444cebad4bf814a5599cb6 (127.24.0.131:37267), 81e26f32389c48e2921a421f51878505 (127.24.0.130:39589)
I20260812 01:19:17.777882 31908 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a89adf633bb4853a2572ca1b06bc615" candidate_uuid: "b678bad42abc4f408e56256d0d11a6f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9b55f3a33444cebad4bf814a5599cb6"
I20260812 01:19:17.777906 31695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a89adf633bb4853a2572ca1b06bc615" candidate_uuid: "b678bad42abc4f408e56256d0d11a6f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81e26f32389c48e2921a421f51878505"
I20260812 01:19:17.777964 31908 raft_consensus.cc:3037] T 4a89adf633bb4853a2572ca1b06bc615 P e9b55f3a33444cebad4bf814a5599cb6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:19:17.777976 31695 raft_consensus.cc:3037] T 4a89adf633bb4853a2572ca1b06bc615 P 81e26f32389c48e2921a421f51878505 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:19:17.778896 31695 raft_consensus.cc:2454] T 4a89adf633bb4853a2572ca1b06bc615 P 81e26f32389c48e2921a421f51878505 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b678bad42abc4f408e56256d0d11a6f1 in term 1.
I20260812 01:19:17.778896 31908 raft_consensus.cc:2454] T 4a89adf633bb4853a2572ca1b06bc615 P e9b55f3a33444cebad4bf814a5599cb6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b678bad42abc4f408e56256d0d11a6f1 in term 1.
I20260812 01:19:17.779104 31377 leader_election.cc:302] T 4a89adf633bb4853a2572ca1b06bc615 P b678bad42abc4f408e56256d0d11a6f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b678bad42abc4f408e56256d0d11a6f1, e9b55f3a33444cebad4bf814a5599cb6; no voters:
I20260812 01:19:17.779213 32025 raft_consensus.cc:2781] T 4a89adf633bb4853a2572ca1b06bc615 P b678bad42abc4f408e56256d0d11a6f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:19:17.779420 32025 raft_consensus.cc:686] T 4a89adf633bb4853a2572ca1b06bc615 P b678bad42abc4f408e56256d0d11a6f1 [term 1 LEADER]: Becoming Leader. State: Replica: b678bad42abc4f408e56256d0d11a6f1, State: Running, Role: LEADER
I20260812 01:19:17.779525 32025 consensus_queue.cc:237] T 4a89adf633bb4853a2572ca1b06bc615 P b678bad42abc4f408e56256d0d11a6f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9b55f3a33444cebad4bf814a5599cb6" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 37267 } } peers { permanent_uuid: "81e26f32389c48e2921a421f51878505" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39589 } } peers { permanent_uuid: "b678bad42abc4f408e56256d0d11a6f1" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 32837 } }
I20260812 01:19:17.780408 31250 catalog_manager.cc:5495] T 4a89adf633bb4853a2572ca1b06bc615 P b678bad42abc4f408e56256d0d11a6f1 reported cstate change: term changed from 0 to 1, leader changed from <none> to b678bad42abc4f408e56256d0d11a6f1 (127.24.0.129). New cstate: current_term: 1 leader_uuid: "b678bad42abc4f408e56256d0d11a6f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9b55f3a33444cebad4bf814a5599cb6" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 37267 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "81e26f32389c48e2921a421f51878505" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39589 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b678bad42abc4f408e56256d0d11a6f1" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 32837 } health_report { overall_health: HEALTHY } } }
I20260812 01:19:17.810576 32000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:17.808116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42810 (local address 127.24.0.129:32837)
0812 01:19:17.808161 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:17.808163 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:17.808182 (+ 19us) server_negotiation.cc:407] Connection header received
0812 01:19:17.808230 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:17.808231 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:17.808252 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:17.808362 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:19:17.808521 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:17.808998 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:17.809772 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:17.809967 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:17.810045 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:17.810227 (+ 182us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:19:17.810333 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:19:17.810434 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:17.810454 (+ 20us) server_negotiation.cc:299] Negotiation successful
0812 01:19:17.810475 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 01:19:17.812619 31908 raft_consensus.cc:1261] T 4a89adf633bb4853a2572ca1b06bc615 P e9b55f3a33444cebad4bf814a5599cb6 [term 1 FOLLOWER]: Refusing update from remote peer b678bad42abc4f408e56256d0d11a6f1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:19:17.812605 31695 raft_consensus.cc:1261] T 4a89adf633bb4853a2572ca1b06bc615 P 81e26f32389c48e2921a421f51878505 [term 1 FOLLOWER]: Refusing update from remote peer b678bad42abc4f408e56256d0d11a6f1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:19:17.813009 32025 consensus_queue.cc:1035] T 4a89adf633bb4853a2572ca1b06bc615 P b678bad42abc4f408e56256d0d11a6f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "81e26f32389c48e2921a421f51878505" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39589 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:19:17.813169 32029 consensus_queue.cc:1035] T 4a89adf633bb4853a2572ca1b06bc615 P b678bad42abc4f408e56256d0d11a6f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9b55f3a33444cebad4bf814a5599cb6" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 37267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:19:17.818377 31246 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57922:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:19:17.818495 31246 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57922:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:19:17.819257 31246 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9dbaddfc899c447ca2731952dd2b3ce9: Sending DeleteTablet for 3 replicas of tablet 4a89adf633bb4853a2572ca1b06bc615
I20260812 01:19:17.819607 31663 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a89adf633bb4853a2572ca1b06bc615 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:19:17 UTC) from {username='kudu', principal='kudu/127.24.0.190@KRBTEST.COM'} at 127.0.0.1:37652
I20260812 01:19:17.819713 31456 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a89adf633bb4853a2572ca1b06bc615 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:19:17 UTC) from {username='kudu', principal='kudu/127.24.0.190@KRBTEST.COM'} at 127.0.0.1:42808
I20260812 01:19:17.819900 31873 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a89adf633bb4853a2572ca1b06bc615 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:19:17 UTC) from {username='kudu', principal='kudu/127.24.0.190@KRBTEST.COM'} at 127.0.0.1:60354
I20260812 01:19:17.819950 32047 tablet_replica.cc:330] T 4a89adf633bb4853a2572ca1b06bc615 P b678bad42abc4f408e56256d0d11a6f1: stopping tablet replica
I20260812 01:19:17.820047 32047 raft_consensus.cc:2229] T 4a89adf633bb4853a2572ca1b06bc615 P b678bad42abc4f408e56256d0d11a6f1 [term 1 LEADER]: Raft consensus shutting down.
I20260812 01:19:17.820147 32049 tablet_replica.cc:330] T 4a89adf633bb4853a2572ca1b06bc615 P e9b55f3a33444cebad4bf814a5599cb6: stopping tablet replica
I20260812 01:19:17.820173 32047 raft_consensus.cc:2258] T 4a89adf633bb4853a2572ca1b06bc615 P b678bad42abc4f408e56256d0d11a6f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:19:17.820340 32046 tablet_replica.cc:330] T 4a89adf633bb4853a2572ca1b06bc615 P 81e26f32389c48e2921a421f51878505: stopping tablet replica
I20260812 01:19:17.820348 32049 raft_consensus.cc:2229] T 4a89adf633bb4853a2572ca1b06bc615 P e9b55f3a33444cebad4bf814a5599cb6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 01:19:17.820458 32049 raft_consensus.cc:2258] T 4a89adf633bb4853a2572ca1b06bc615 P e9b55f3a33444cebad4bf814a5599cb6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:19:17.820484 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 31353
I20260812 01:19:17.820670 32046 raft_consensus.cc:2229] T 4a89adf633bb4853a2572ca1b06bc615 P 81e26f32389c48e2921a421f51878505 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 01:19:17.820768 32046 raft_consensus.cc:2258] T 4a89adf633bb4853a2572ca1b06bc615 P 81e26f32389c48e2921a421f51878505 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:19:17.821350 32049 ts_tablet_manager.cc:1905] T 4a89adf633bb4853a2572ca1b06bc615 P e9b55f3a33444cebad4bf814a5599cb6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:19:17.823047 32049 ts_tablet_manager.cc:1918] T 4a89adf633bb4853a2572ca1b06bc615 P e9b55f3a33444cebad4bf814a5599cb6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 01:19:17.823109 32049 log.cc:1196] T 4a89adf633bb4853a2572ca1b06bc615 P e9b55f3a33444cebad4bf814a5599cb6: Deleting WAL directory at /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786497534195564-24578-0/minicluster-data/ts-2/wal/wals/4a89adf633bb4853a2572ca1b06bc615
I20260812 01:19:17.823354 32049 ts_tablet_manager.cc:1939] T 4a89adf633bb4853a2572ca1b06bc615 P e9b55f3a33444cebad4bf814a5599cb6: Deleting consensus metadata
I20260812 01:19:17.823834 31216 catalog_manager.cc:4841] TS e9b55f3a33444cebad4bf814a5599cb6 (127.24.0.131:37267): tablet 4a89adf633bb4853a2572ca1b06bc615 (table test-table [id=8c67db97cb7348379bc0ca1dbe5f8bcf]) successfully deleted
I20260812 01:19:17.824378 32046 ts_tablet_manager.cc:1905] T 4a89adf633bb4853a2572ca1b06bc615 P 81e26f32389c48e2921a421f51878505: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:19:17.826906 32046 ts_tablet_manager.cc:1918] T 4a89adf633bb4853a2572ca1b06bc615 P 81e26f32389c48e2921a421f51878505: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 01:19:17.827075 32046 log.cc:1196] T 4a89adf633bb4853a2572ca1b06bc615 P 81e26f32389c48e2921a421f51878505: Deleting WAL directory at /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786497534195564-24578-0/minicluster-data/ts-1/wal/wals/4a89adf633bb4853a2572ca1b06bc615
I20260812 01:19:17.827345 32046 ts_tablet_manager.cc:1939] T 4a89adf633bb4853a2572ca1b06bc615 P 81e26f32389c48e2921a421f51878505: Deleting consensus metadata
I20260812 01:19:17.827764 31221 catalog_manager.cc:4841] TS 81e26f32389c48e2921a421f51878505 (127.24.0.130:39589): tablet 4a89adf633bb4853a2572ca1b06bc615 (table test-table [id=8c67db97cb7348379bc0ca1dbe5f8bcf]) successfully deleted
W20260812 01:19:17.831629 31219 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.0.129:32837 (error 108)
I20260812 01:19:17.831818 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 31568
I20260812 01:19:17.836112 31998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:17.835963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.129:32837 (local address 127.0.0.1:42816)
0812 01:19:17.836018 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:17.836070 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.129:32837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260812 01:19:17.836341 31219 catalog_manager.cc:4584] TS b678bad42abc4f408e56256d0d11a6f1 (127.24.0.129:32837): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4a89adf633bb4853a2572ca1b06bc615: Network error: Client connection negotiation failed: client connection to 127.24.0.129:32837: connect: Connection refused (error 111)
I20260812 01:19:17.844780 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 31776
I20260812 01:19:17.851378 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 31189
2026-08-12T01:19:17Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2425 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-12T01:19:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:19:17Z Disabled control of system clock
I20260812 01:19:17.889765 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786497534195564-24578-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786497534195564-24578-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786497534195564-24578-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.190:33889
--webserver_interface=127.24.0.190
--webserver_port=0
--builtin_ntp_servers=127.24.0.148:34641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.0.190:33889
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260812 01:19:17.971954 32057 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:19:17.972154 32057 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:19:17.972175 32057 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:19:17.973354 32057 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:19:17.973397 32057 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:19:17.973408 32057 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:19:17.973421 32057 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:19:17.973476 32057 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:19:17.974778 32057 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:34641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786497534195564-24578-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786497534195564-24578-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.0.190:33889
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.190:33889
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.0.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32057
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786497534195564-24578-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:19:17.975026 32057 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:19:17.975315 32057 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:19:17.979982 32069 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:17.980006 32063 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:17.980024 32066 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:19:17.980091 32057 server_base.cc:1034] running on GCE node
I20260812 01:19:17.980552 32057 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:19:17.980845 32057 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:19:17.982019 32057 hybrid_clock.cc:648] HybridClock initialized: now 1786497557982014 us; error 36 us; skew 500 ppm
I20260812 01:19:17.983214 32057 webserver.cc:460] Webserver started at http://127.24.0.190:32781/ using document root <none> and password file <none>
I20260812 01:19:17.983445 32057 fs_manager.cc:359] Metadata directory not provided
I20260812 01:19:17.983515 32057 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:19:17.983685 32057 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:19:17.984865 32057 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786497534195564-24578-0/minicluster-data/master-0/data/instance:
uuid: "e31aba4353e746e4bb67c46bfc6e9e59"
format_stamp: "Formatted at 2026-08-12 01:19:17 on dist-test-slave-t3q3"
server_key: "ee7dd0beaad719431384c51f259f5260"
server_key_iv: "1f1fb286d7a50dc14572f8c56e52e969"
server_key_version: "encryptionkey@0"
I20260812 01:19:17.985226 32057 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786497534195564-24578-0/minicluster-data/master-0/wal/instance:
uuid: "e31aba4353e746e4bb67c46bfc6e9e59"
format_stamp: "Formatted at 2026-08-12 01:19:17 on dist-test-slave-t3q3"
server_key: "ee7dd0beaad719431384c51f259f5260"
server_key_iv: "1f1fb286d7a50dc14572f8c56e52e969"
server_key_version: "encryptionkey@0"
I20260812 01:19:17.986503 32057 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 01:19:17.987210 32077 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:17.987383 32057 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 01:19:17.987483 32057 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786497534195564-24578-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786497534195564-24578-0/minicluster-data/master-0/wal
uuid: "e31aba4353e746e4bb67c46bfc6e9e59"
format_stamp: "Formatted at 2026-08-12 01:19:17 on dist-test-slave-t3q3"
server_key: "ee7dd0beaad719431384c51f259f5260"
server_key_iv: "1f1fb286d7a50dc14572f8c56e52e969"
server_key_version: "encryptionkey@0"
I20260812 01:19:17.987564 32057 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786497534195564-24578-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786497534195564-24578-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786497534195564-24578-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:19:17.990931 32057 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:19:17.991187 32057 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:19:17.991328 32057 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:19:17.995456 32057 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.190:33889
I20260812 01:19:17.995489 32153 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.190:33889 every 8 connection(s)
I20260812 01:19:17.995800 32057 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
I20260812 01:19:17.996477 32154 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:19:17.999212 32154 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e31aba4353e746e4bb67c46bfc6e9e59: Bootstrap starting.
I20260812 01:19:17.999807 32154 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e31aba4353e746e4bb67c46bfc6e9e59: Neither blocks nor log segments found. Creating new log.
I20260812 01:19:18.000156 32154 log.cc:824] T 00000000000000000000000000000000 P e31aba4353e746e4bb67c46bfc6e9e59: Log is configured to *not* fsync() on all Append() calls
I20260812 01:19:18.000839 32154 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e31aba4353e746e4bb67c46bfc6e9e59: No bootstrap required, opened a new log
I20260812 01:19:18.002413 32154 raft_consensus.cc:348] T 00000000000000000000000000000000 P e31aba4353e746e4bb67c46bfc6e9e59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e31aba4353e746e4bb67c46bfc6e9e59" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 33889 } }
I20260812 01:19:18.002544 32154 raft_consensus.cc:374] T 00000000000000000000000000000000 P e31aba4353e746e4bb67c46bfc6e9e59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:19:18.002592 32154 raft_consensus.cc:729] T 00000000000000000000000000000000 P e31aba4353e746e4bb67c46bfc6e9e59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e31aba4353e746e4bb67c46bfc6e9e59, State: Initialized, Role: FOLLOWER
I20260812 01:19:18.002740 32154 consensus_queue.cc:260] T 00000000000000000000000000000000 P e31aba4353e746e4bb67c46bfc6e9e59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e31aba4353e746e4bb67c46bfc6e9e59" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 33889 } }
I20260812 01:19:18.002803 32154 raft_consensus.cc:388] T 00000000000000000000000000000000 P e31aba4353e746e4bb67c46bfc6e9e59 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:19:18.002846 32154 raft_consensus.cc:482] T 00000000000000000000000000000000 P e31aba4353e746e4bb67c46bfc6e9e59 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:19:18.002897 32154 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e31aba4353e746e4bb67c46bfc6e9e59 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:19:18.003440 32154 raft_consensus.cc:504] T 00000000000000000000000000000000 P e31aba4353e746e4bb67c46bfc6e9e59 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e31aba4353e746e4bb67c46bfc6e9e59" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 33889 } }
I20260812 01:19:18.003556 32154 leader_election.cc:302] T 00000000000000000000000000000000 P e31aba4353e746e4bb67c46bfc6e9e59 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e31aba4353e746e4bb67c46bfc6e9e59; no voters:
I20260812 01:19:18.003737 32154 leader_election.cc:288] T 00000000000000000000000000000000 P e31aba4353e746e4bb67c46bfc6e9e59 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 01:19:18.003783 32162 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e31aba4353e746e4bb67c46bfc6e9e59 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:19:18.003966 32162 raft_consensus.cc:686] T 00000000000000000000000000000000 P e31aba4353e746e4bb67c46bfc6e9e59 [term 1 LEADER]: Becoming Leader. State: Replica: e31aba4353e746e4bb67c46bfc6e9e59, State: Running, Role: LEADER
I20260812 01:19:18.004009 32154 sys_catalog.cc:564] T 00000000000000000000000000000000 P e31aba4353e746e4bb67c46bfc6e9e59 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:19:18.004218 32162 consensus_queue.cc:237] T 00000000000000000000000000000000 P e31aba4353e746e4bb67c46bfc6e9e59 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e31aba4353e746e4bb67c46bfc6e9e59" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 33889 } }
I20260812 01:19:18.004532 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 32057
I20260812 01:19:18.004652 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786497534195564-24578-0/minicluster-data/master-0/wal/instance
I20260812 01:19:18.004632 32165 sys_catalog.cc:455] T 00000000000000000000000000000000 P e31aba4353e746e4bb67c46bfc6e9e59 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e31aba4353e746e4bb67c46bfc6e9e59" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e31aba4353e746e4bb67c46bfc6e9e59" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 33889 } } }
I20260812 01:19:18.004737 32165 sys_catalog.cc:458] T 00000000000000000000000000000000 P e31aba4353e746e4bb67c46bfc6e9e59 [sys.catalog]: This master's current role is: LEADER
I20260812 01:19:18.004868 24578 external_mini_cluster.cc:1428] Setting key c457fa9480fd336939aeef350fb5784a
I20260812 01:19:18.005025 32167 sys_catalog.cc:455] T 00000000000000000000000000000000 P e31aba4353e746e4bb67c46bfc6e9e59 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e31aba4353e746e4bb67c46bfc6e9e59. Latest consensus state: current_term: 1 leader_uuid: "e31aba4353e746e4bb67c46bfc6e9e59" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e31aba4353e746e4bb67c46bfc6e9e59" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 33889 } } }
I20260812 01:19:18.005123 32167 sys_catalog.cc:458] T 00000000000000000000000000000000 P e31aba4353e746e4bb67c46bfc6e9e59 [sys.catalog]: This master's current role is: LEADER
I20260812 01:19:18.007092 32181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:18.005381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34534 (local address 127.24.0.190:33889)
0812 01:19:18.005566 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:18.005576 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:18.005617 (+ 41us) server_negotiation.cc:407] Connection header received
0812 01:19:18.005773 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:18.005780 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:18.005827 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:18.005998 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:18.006654 (+ 656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:18.006658 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:18.006658 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:19:18.006670 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:18.006711 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:19:18.006724 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:18.006810 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:18.006884 (+ 74us) server_negotiation.cc:299] Negotiation successful
0812 01:19:18.006941 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":81,"threads_started":1}
W20260812 01:19:18.007102 32189 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e31aba4353e746e4bb67c46bfc6e9e59: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 01:19:18.007174 32189 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 01:19:18.007227 32170 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:19:18.007362 32170 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:19:18.008821 32170 catalog_manager.cc:1347] Generated new cluster ID: f8324902ff4141dca681c9559b2fa412
I20260812 01:19:18.008889 32170 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 01:19:18.018479 32170 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:19:18.018626 32170 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:19:18.021616 32170 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e31aba4353e746e4bb67c46bfc6e9e59: Generated new TSK 0
I20260812 01:19:18.021807 32170 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 01:19:18.025293 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786497534195564-24578-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786497534195564-24578-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786497534195564-24578-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786497534195564-24578-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.0.129:0
--local_ip_for_outbound_sockets=127.24.0.129
--webserver_interface=127.24.0.129
--webserver_port=0
--tserver_master_addrs=127.24.0.190:33889
--builtin_ntp_servers=127.24.0.148:34641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260812 01:19:18.127357 32192 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:19:18.127588 32192 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:19:18.127612 32192 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:19:18.128835 32192 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:19:18.128878 32192 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:19:18.128904 32192 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.0.129
I20260812 01:19:18.130285 32192 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:34641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786497534195564-24578-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786497534195564-24578-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.0.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786497534195564-24578-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.0.129
--webserver_port=0
--tserver_master_addrs=127.24.0.190:33889
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32192
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.0.129
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786497534195564-24578-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:19:18.130553 32192 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:19:18.130892 32192 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:19:18.136902 32199 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:18.136884 32201 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:18.136883 32206 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:19:18.137367 32192 server_base.cc:1034] running on GCE node
I20260812 01:19:18.137611 32192 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:19:18.137892 32192 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:19:18.139075 32192 hybrid_clock.cc:648] HybridClock initialized: now 1786497558139068 us; error 45 us; skew 500 ppm
I20260812 01:19:18.140725 32192 webserver.cc:460] Webserver started at http://127.24.0.129:33039/ using document root <none> and password file <none>
I20260812 01:19:18.141022 32192 fs_manager.cc:359] Metadata directory not provided
I20260812 01:19:18.141098 32192 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:19:18.141224 32192 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:19:18.142679 32192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786497534195564-24578-0/minicluster-data/ts-0/data/instance:
uuid: "3a24ca4abd994042a92e9b1d2577152a"
format_stamp: "Formatted at 2026-08-12 01:19:18 on dist-test-slave-t3q3"
server_key: "6718fa8ec9cabc70db063c6eba90b897"
server_key_iv: "63cf2cabc5a0737ce37d05b69495ffdd"
server_key_version: "encryptionkey@0"
I20260812 01:19:18.143139 32192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786497534195564-24578-0/minicluster-data/ts-0/wal/instance:
uuid: "3a24ca4abd994042a92e9b1d2577152a"
format_stamp: "Formatted at 2026-08-12 01:19:18 on dist-test-slave-t3q3"
server_key: "6718fa8ec9cabc70db063c6eba90b897"
server_key_iv: "63cf2cabc5a0737ce37d05b69495ffdd"
server_key_version: "encryptionkey@0"
I20260812 01:19:18.145138 32192 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.003s
I20260812 01:19:18.146308 32222 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:18.146551 32192 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:19:18.146661 32192 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786497534195564-24578-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786497534195564-24578-0/minicluster-data/ts-0/wal
uuid: "3a24ca4abd994042a92e9b1d2577152a"
format_stamp: "Formatted at 2026-08-12 01:19:18 on dist-test-slave-t3q3"
server_key: "6718fa8ec9cabc70db063c6eba90b897"
server_key_iv: "63cf2cabc5a0737ce37d05b69495ffdd"
server_key_version: "encryptionkey@0"
I20260812 01:19:18.146745 32192 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786497534195564-24578-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786497534195564-24578-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786497534195564-24578-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:19:18.152127 32192 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:19:18.152443 32192 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:19:18.152591 32192 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:19:18.152806 32192 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:19:18.153177 32192 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:19:18.153232 32192 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:18.153261 32192 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:19:18.153280 32192 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:18.163056 32192 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.129:40381
I20260812 01:19:18.163094 32404 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.129:40381 every 8 connection(s)
I20260812 01:19:18.163554 32192 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786497534195564-24578-0/minicluster-data/ts-0/data/info.pb
I20260812 01:19:18.167068 32181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:18.164537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.129:53491 (local address 127.24.0.190:33889)
0812 01:19:18.164617 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:18.164619 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:18.164922 (+ 303us) server_negotiation.cc:407] Connection header received
0812 01:19:18.165465 (+ 543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:18.165466 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:18.165474 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:18.165542 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:18.166789 (+ 1247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:18.166791 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:18.166791 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:19:18.166796 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:18.166813 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:19:18.166818 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:18.166878 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:18.166979 (+ 101us) server_negotiation.cc:299] Negotiation successful
0812 01:19:18.166996 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 01:19:18.167282 32413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:18.164545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:33889 (local address 127.24.0.129:53491)
0812 01:19:18.164777 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:18.164817 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:18.165351 (+ 534us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:18.165622 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:18.165625 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:18.165633 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:18.166617 (+ 984us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 01:19:18.166618 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:18.166648 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 01:19:18.166686 (+ 38us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:19:18.166884 (+ 198us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:19:18.166890 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:19:18.166893 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 01:19:18.166989 (+ 96us) client_negotiation.cc:240] Negotiation successful
0812 01:19:18.167056 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":112,"threads_started":1}
I20260812 01:19:18.167963 32405 heartbeater.cc:347] Connected to a master server at 127.24.0.190:33889
I20260812 01:19:18.168092 32405 heartbeater.cc:464] Registering TS with master...
I20260812 01:19:18.168255 32405 heartbeater.cc:511] Master 127.24.0.190:33889 requested a full tablet report, sending...
I20260812 01:19:18.168814 32106 ts_manager.cc:194] Registered new tserver with Master: 3a24ca4abd994042a92e9b1d2577152a (127.24.0.129:40381)
I20260812 01:19:18.171940 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 32192
I20260812 01:19:18.172067 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786497534195564-24578-0/minicluster-data/ts-0/wal/instance
I20260812 01:19:18.172281 24578 external_mini_cluster.cc:1428] Setting key 4d32d0a4e3e0965af12c164490ba92bd
I20260812 01:19:18.173408 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786497534195564-24578-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786497534195564-24578-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786497534195564-24578-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786497534195564-24578-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.0.130:0
--local_ip_for_outbound_sockets=127.24.0.130
--webserver_interface=127.24.0.130
--webserver_port=0
--tserver_master_addrs=127.24.0.190:33889
--builtin_ntp_servers=127.24.0.148:34641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260812 01:19:18.258823 32414 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:19:18.259028 32414 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:19:18.259052 32414 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:19:18.260159 32414 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:19:18.260192 32414 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:19:18.260217 32414 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.0.130
I20260812 01:19:18.261475 32414 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:34641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786497534195564-24578-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786497534195564-24578-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.0.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786497534195564-24578-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.0.130
--webserver_port=0
--tserver_master_addrs=127.24.0.190:33889
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32414
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.0.130
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786497534195564-24578-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:19:18.261663 32414 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:19:18.261895 32414 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:19:18.266940 32422 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:18.267057 32420 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:18.266912 32425 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:19:18.267204 32414 server_base.cc:1034] running on GCE node
I20260812 01:19:18.267432 32414 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:19:18.267669 32414 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:19:18.268831 32414 hybrid_clock.cc:648] HybridClock initialized: now 1786497558268815 us; error 29 us; skew 500 ppm
I20260812 01:19:18.270259 32414 webserver.cc:460] Webserver started at http://127.24.0.130:44513/ using document root <none> and password file <none>
I20260812 01:19:18.270519 32414 fs_manager.cc:359] Metadata directory not provided
I20260812 01:19:18.270574 32414 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:19:18.270758 32414 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:19:18.271847 32414 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786497534195564-24578-0/minicluster-data/ts-1/data/instance:
uuid: "9d5ed02e09f64c00bba7c9e0c57a9f63"
format_stamp: "Formatted at 2026-08-12 01:19:18 on dist-test-slave-t3q3"
server_key: "5b1e1c44330a6eb268430c320c3b4796"
server_key_iv: "8a7b6c4a9f13b0851b0191d31f869fee"
server_key_version: "encryptionkey@0"
I20260812 01:19:18.272332 32414 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786497534195564-24578-0/minicluster-data/ts-1/wal/instance:
uuid: "9d5ed02e09f64c00bba7c9e0c57a9f63"
format_stamp: "Formatted at 2026-08-12 01:19:18 on dist-test-slave-t3q3"
server_key: "5b1e1c44330a6eb268430c320c3b4796"
server_key_iv: "8a7b6c4a9f13b0851b0191d31f869fee"
server_key_version: "encryptionkey@0"
I20260812 01:19:18.273761 32414 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 01:19:18.274567 32441 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:18.274741 32414 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 01:19:18.274847 32414 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786497534195564-24578-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786497534195564-24578-0/minicluster-data/ts-1/wal
uuid: "9d5ed02e09f64c00bba7c9e0c57a9f63"
format_stamp: "Formatted at 2026-08-12 01:19:18 on dist-test-slave-t3q3"
server_key: "5b1e1c44330a6eb268430c320c3b4796"
server_key_iv: "8a7b6c4a9f13b0851b0191d31f869fee"
server_key_version: "encryptionkey@0"
I20260812 01:19:18.274930 32414 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786497534195564-24578-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786497534195564-24578-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786497534195564-24578-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:19:18.278407 32414 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:19:18.278690 32414 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:19:18.278833 32414 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:19:18.279057 32414 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:19:18.279333 32414 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:19:18.279371 32414 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:18.279390 32414 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:19:18.279402 32414 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:18.285305 32414 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.130:45861
I20260812 01:19:18.285348 32614 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.130:45861 every 8 connection(s)
I20260812 01:19:18.285687 32414 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786497534195564-24578-0/minicluster-data/ts-1/data/info.pb
I20260812 01:19:18.288940 32181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:18.286399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.130:59365 (local address 127.24.0.190:33889)
0812 01:19:18.286463 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:18.286465 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:18.286733 (+ 268us) server_negotiation.cc:407] Connection header received
0812 01:19:18.287254 (+ 521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:18.287255 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:18.287262 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:18.287325 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:18.288658 (+ 1333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:18.288660 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:18.288661 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:19:18.288666 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:18.288684 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:19:18.288688 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:18.288749 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:18.288854 (+ 105us) server_negotiation.cc:299] Negotiation successful
0812 01:19:18.288870 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260812 01:19:18.289142 32619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:18.286382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:33889 (local address 127.24.0.130:59365)
0812 01:19:18.286611 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:18.286645 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:18.287144 (+ 499us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:18.287393 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:18.287399 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:18.287413 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:18.288466 (+ 1053us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 01:19:18.288467 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:18.288497 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 01:19:18.288526 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:19:18.288759 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:19:18.288765 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:19:18.288768 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 01:19:18.288864 (+ 96us) client_negotiation.cc:240] Negotiation successful
0812 01:19:18.288927 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":95,"threads_started":1}
I20260812 01:19:18.289216 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 32414
I20260812 01:19:18.289314 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786497534195564-24578-0/minicluster-data/ts-1/wal/instance
I20260812 01:19:18.289548 24578 external_mini_cluster.cc:1428] Setting key 7134366e192044984269261826116dbc
I20260812 01:19:18.289800 32615 heartbeater.cc:347] Connected to a master server at 127.24.0.190:33889
I20260812 01:19:18.289908 32615 heartbeater.cc:464] Registering TS with master...
I20260812 01:19:18.290074 32615 heartbeater.cc:511] Master 127.24.0.190:33889 requested a full tablet report, sending...
I20260812 01:19:18.290483 32106 ts_manager.cc:194] Registered new tserver with Master: 9d5ed02e09f64c00bba7c9e0c57a9f63 (127.24.0.130:45861)
I20260812 01:19:18.290645 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786497534195564-24578-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786497534195564-24578-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786497534195564-24578-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786497534195564-24578-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.0.131:0
--local_ip_for_outbound_sockets=127.24.0.131
--webserver_interface=127.24.0.131
--webserver_port=0
--tserver_master_addrs=127.24.0.190:33889
--builtin_ntp_servers=127.24.0.148:34641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260812 01:19:18.374336 32621 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:19:18.374553 32621 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:19:18.374576 32621 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:19:18.375734 32621 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:19:18.375777 32621 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:19:18.375801 32621 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.0.131
I20260812 01:19:18.377295 32621 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:34641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786497534195564-24578-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786497534195564-24578-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.0.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786497534195564-24578-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.0.131
--webserver_port=0
--tserver_master_addrs=127.24.0.190:33889
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32621
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.0.131
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786497534195564-24578-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:19:18.377542 32621 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:19:18.377846 32621 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:19:18.382890 32634 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:18.382867 32637 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:18.382875 32631 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:19:18.383215 32621 server_base.cc:1034] running on GCE node
I20260812 01:19:18.383448 32621 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:19:18.383711 32621 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:19:18.384881 32621 hybrid_clock.cc:648] HybridClock initialized: now 1786497558384876 us; error 38 us; skew 500 ppm
I20260812 01:19:18.386181 32621 webserver.cc:460] Webserver started at http://127.24.0.131:38771/ using document root <none> and password file <none>
I20260812 01:19:18.386452 32621 fs_manager.cc:359] Metadata directory not provided
I20260812 01:19:18.386521 32621 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:19:18.386662 32621 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:19:18.387790 32621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786497534195564-24578-0/minicluster-data/ts-2/data/instance:
uuid: "a31cffc1ba2547149c1889158e1650f6"
format_stamp: "Formatted at 2026-08-12 01:19:18 on dist-test-slave-t3q3"
server_key: "453f328a0a6f850761485bc7c5c14a25"
server_key_iv: "4a5479eac10d727bdb98553bfb77ac23"
server_key_version: "encryptionkey@0"
I20260812 01:19:18.388230 32621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786497534195564-24578-0/minicluster-data/ts-2/wal/instance:
uuid: "a31cffc1ba2547149c1889158e1650f6"
format_stamp: "Formatted at 2026-08-12 01:19:18 on dist-test-slave-t3q3"
server_key: "453f328a0a6f850761485bc7c5c14a25"
server_key_iv: "4a5479eac10d727bdb98553bfb77ac23"
server_key_version: "encryptionkey@0"
I20260812 01:19:18.389590 32621 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 01:19:18.390393 32644 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:18.390595 32621 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 01:19:18.390698 32621 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786497534195564-24578-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786497534195564-24578-0/minicluster-data/ts-2/wal
uuid: "a31cffc1ba2547149c1889158e1650f6"
format_stamp: "Formatted at 2026-08-12 01:19:18 on dist-test-slave-t3q3"
server_key: "453f328a0a6f850761485bc7c5c14a25"
server_key_iv: "4a5479eac10d727bdb98553bfb77ac23"
server_key_version: "encryptionkey@0"
I20260812 01:19:18.390789 32621 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786497534195564-24578-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786497534195564-24578-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786497534195564-24578-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:19:18.394275 32621 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:19:18.394539 32621 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:19:18.394690 32621 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:19:18.394909 32621 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:19:18.395231 32621 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:19:18.395287 32621 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:18.395326 32621 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:19:18.395354 32621 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:18.401253 32621 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.131:36041
I20260812 01:19:18.401290 353 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.131:36041 every 8 connection(s)
I20260812 01:19:18.401645 32621 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786497534195564-24578-0/minicluster-data/ts-2/data/info.pb
I20260812 01:19:18.404767 32181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:18.402408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.131:60299 (local address 127.24.0.190:33889)
0812 01:19:18.402467 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:18.402469 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:18.402673 (+ 204us) server_negotiation.cc:407] Connection header received
0812 01:19:18.403106 (+ 433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:18.403107 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:18.403112 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:18.403183 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:18.404531 (+ 1348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:18.404532 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:18.404533 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:19:18.404536 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:18.404549 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:19:18.404553 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:18.404636 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:18.404696 (+ 60us) server_negotiation.cc:299] Negotiation successful
0812 01:19:18.404709 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260812 01:19:18.404983 356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:18.402353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:33889 (local address 127.24.0.131:60299)
0812 01:19:18.402538 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:18.402577 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:18.403019 (+ 442us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:18.403254 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:18.403257 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:18.403265 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:18.404391 (+ 1126us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 01:19:18.404392 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:18.404421 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 01:19:18.404450 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:19:18.404641 (+ 191us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:19:18.404645 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:19:18.404647 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 01:19:18.404706 (+ 59us) client_negotiation.cc:240] Negotiation successful
0812 01:19:18.404755 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":94,"threads_started":1}
I20260812 01:19:18.405521 354 heartbeater.cc:347] Connected to a master server at 127.24.0.190:33889
I20260812 01:19:18.405623 354 heartbeater.cc:464] Registering TS with master...
I20260812 01:19:18.405783 354 heartbeater.cc:511] Master 127.24.0.190:33889 requested a full tablet report, sending...
I20260812 01:19:18.406206 32106 ts_manager.cc:194] Registered new tserver with Master: a31cffc1ba2547149c1889158e1650f6 (127.24.0.131:36041)
I20260812 01:19:18.406282 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 32621
I20260812 01:19:18.406391 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786497534195564-24578-0/minicluster-data/ts-2/wal/instance
I20260812 01:19:18.406589 24578 external_mini_cluster.cc:1428] Setting key 6f1518a02045af2d4b6271edefeb600f
I20260812 01:19:18.407349 24578 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 01:19:18.409771 32181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:18.408708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34548 (local address 127.24.0.190:33889)
0812 01:19:18.408746 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:18.408748 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:18.408887 (+ 139us) server_negotiation.cc:407] Connection header received
0812 01:19:18.408914 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:18.408915 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:18.408919 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:18.408964 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:18.409567 (+ 603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:18.409568 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:18.409568 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:19:18.409571 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:18.409582 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:19:18.409585 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:18.409622 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:18.409657 (+ 35us) server_negotiation.cc:299] Negotiation successful
0812 01:19:18.409677 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260812 01:19:18.411492 32106 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34548:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 01:19:18.413136 32106 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 01:19:18.416980 373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:18.415307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.131:36041 (local address 127.0.0.1:56882)
0812 01:19:18.415662 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:18.415705 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:18.415769 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:18.415971 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:18.415977 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:18.416046 (+ 69us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:18.416560 (+ 514us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 01:19:18.416561 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:18.416581 (+ 20us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 01:19:18.416601 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:19:18.416831 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:19:18.416838 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:19:18.416842 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 01:19:18.416902 (+ 60us) client_negotiation.cc:240] Negotiation successful
0812 01:19:18.416916 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":93,"threads_started":1}
I20260812 01:19:18.417011 371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:18.415469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56882 (local address 127.24.0.131:36041)
0812 01:19:18.415629 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:18.415633 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:18.415831 (+ 198us) server_negotiation.cc:407] Connection header received
0812 01:19:18.415864 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:18.415866 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:18.415873 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:18.415953 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:18.416699 (+ 746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:18.416706 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:18.416706 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:19:18.416711 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:18.416748 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:19:18.416772 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:18.416838 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:18.416920 (+ 82us) server_negotiation.cc:299] Negotiation successful
0812 01:19:18.416935 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":77,"threads_started":1}
I20260812 01:19:18.417309 369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:18.415519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.130:45861 (local address 127.0.0.1:47822)
0812 01:19:18.415863 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:18.415872 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:18.415918 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:18.416465 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:18.416467 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:18.416469 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:18.416887 (+ 418us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 01:19:18.416887 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:18.416894 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 01:19:18.416905 (+ 11us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:19:18.417169 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:19:18.417170 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:19:18.417171 (+ 1us) client_negotiation.cc:769] Sending connection context
0812 01:19:18.417228 (+ 57us) client_negotiation.cc:240] Negotiation successful
0812 01:19:18.417241 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":76,"threads_started":1}
I20260812 01:19:18.417366 374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:18.415648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47822 (local address 127.24.0.130:45861)
0812 01:19:18.415846 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:18.415850 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:18.415910 (+ 60us) server_negotiation.cc:407] Connection header received
0812 01:19:18.415982 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:18.415986 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:18.415995 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:18.416091 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:18.416991 (+ 900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:18.416998 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:18.416998 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:19:18.417003 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:18.417043 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:19:18.417072 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:18.417137 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:18.417244 (+ 107us) server_negotiation.cc:299] Negotiation successful
0812 01:19:18.417262 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":88,"threads_started":1}
I20260812 01:19:18.417495 368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:18.415307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.129:40381 (local address 127.0.0.1:54380)
0812 01:19:18.415675 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:18.415705 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:18.415971 (+ 266us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:18.416671 (+ 700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:18.416672 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:18.416674 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:18.417099 (+ 425us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 01:19:18.417099 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:18.417106 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 01:19:18.417119 (+ 13us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:19:18.417342 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:19:18.417343 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:19:18.417343 (+ 0us) client_negotiation.cc:769] Sending connection context
0812 01:19:18.417419 (+ 76us) client_negotiation.cc:240] Negotiation successful
0812 01:19:18.417433 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":347,"mutex_wait_us":4,"thread_start_us":51,"threads_started":1}
I20260812 01:19:18.417611 370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:18.415401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54380 (local address 127.24.0.129:40381)
0812 01:19:18.415607 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:18.415613 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:18.415805 (+ 192us) server_negotiation.cc:407] Connection header received
0812 01:19:18.416075 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:18.416084 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:18.416094 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:18.416148 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:18.417189 (+ 1041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:18.417197 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:18.417197 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:19:18.417201 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:18.417235 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:19:18.417258 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:18.417316 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:18.417499 (+ 183us) server_negotiation.cc:299] Negotiation successful
0812 01:19:18.417518 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":87,"threads_started":1}
I20260812 01:19:18.417989 32716 tablet_service.cc:1467] Processing CreateTablet for tablet f5df26fb669845569eab70987d6af066 (DEFAULT_TABLE table=test-table [id=13286eb0a16a41239f40920b13569fbc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:19:18.418221 32304 tablet_service.cc:1467] Processing CreateTablet for tablet f5df26fb669845569eab70987d6af066 (DEFAULT_TABLE table=test-table [id=13286eb0a16a41239f40920b13569fbc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:19:18.418368 32716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5df26fb669845569eab70987d6af066. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:19:18.418524 32304 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5df26fb669845569eab70987d6af066. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:19:18.420760 32514 tablet_service.cc:1467] Processing CreateTablet for tablet f5df26fb669845569eab70987d6af066 (DEFAULT_TABLE table=test-table [id=13286eb0a16a41239f40920b13569fbc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:19:18.421111 32514 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5df26fb669845569eab70987d6af066. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:19:18.421756 378 tablet_bootstrap.cc:492] T f5df26fb669845569eab70987d6af066 P a31cffc1ba2547149c1889158e1650f6: Bootstrap starting.
I20260812 01:19:18.422013 375 tablet_bootstrap.cc:492] T f5df26fb669845569eab70987d6af066 P 3a24ca4abd994042a92e9b1d2577152a: Bootstrap starting.
I20260812 01:19:18.422585 378 tablet_bootstrap.cc:654] T f5df26fb669845569eab70987d6af066 P a31cffc1ba2547149c1889158e1650f6: Neither blocks nor log segments found. Creating new log.
I20260812 01:19:18.422659 375 tablet_bootstrap.cc:654] T f5df26fb669845569eab70987d6af066 P 3a24ca4abd994042a92e9b1d2577152a: Neither blocks nor log segments found. Creating new log.
I20260812 01:19:18.422952 375 log.cc:824] T f5df26fb669845569eab70987d6af066 P 3a24ca4abd994042a92e9b1d2577152a: Log is configured to *not* fsync() on all Append() calls
I20260812 01:19:18.422940 378 log.cc:824] T f5df26fb669845569eab70987d6af066 P a31cffc1ba2547149c1889158e1650f6: Log is configured to *not* fsync() on all Append() calls
I20260812 01:19:18.423925 375 tablet_bootstrap.cc:492] T f5df26fb669845569eab70987d6af066 P 3a24ca4abd994042a92e9b1d2577152a: No bootstrap required, opened a new log
I20260812 01:19:18.423977 378 tablet_bootstrap.cc:492] T f5df26fb669845569eab70987d6af066 P a31cffc1ba2547149c1889158e1650f6: No bootstrap required, opened a new log
I20260812 01:19:18.424000 375 ts_tablet_manager.cc:1397] T f5df26fb669845569eab70987d6af066 P 3a24ca4abd994042a92e9b1d2577152a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 01:19:18.424086 378 ts_tablet_manager.cc:1397] T f5df26fb669845569eab70987d6af066 P a31cffc1ba2547149c1889158e1650f6: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 01:19:18.424551 384 tablet_bootstrap.cc:492] T f5df26fb669845569eab70987d6af066 P 9d5ed02e09f64c00bba7c9e0c57a9f63: Bootstrap starting.
I20260812 01:19:18.425321 384 tablet_bootstrap.cc:654] T f5df26fb669845569eab70987d6af066 P 9d5ed02e09f64c00bba7c9e0c57a9f63: Neither blocks nor log segments found. Creating new log.
I20260812 01:19:18.425710 384 log.cc:824] T f5df26fb669845569eab70987d6af066 P 9d5ed02e09f64c00bba7c9e0c57a9f63: Log is configured to *not* fsync() on all Append() calls
I20260812 01:19:18.426164 378 raft_consensus.cc:348] T f5df26fb669845569eab70987d6af066 P a31cffc1ba2547149c1889158e1650f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a31cffc1ba2547149c1889158e1650f6" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 36041 } } peers { permanent_uuid: "9d5ed02e09f64c00bba7c9e0c57a9f63" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 45861 } } peers { permanent_uuid: "3a24ca4abd994042a92e9b1d2577152a" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 40381 } }
I20260812 01:19:18.426368 378 raft_consensus.cc:374] T f5df26fb669845569eab70987d6af066 P a31cffc1ba2547149c1889158e1650f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:19:18.426417 378 raft_consensus.cc:729] T f5df26fb669845569eab70987d6af066 P a31cffc1ba2547149c1889158e1650f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a31cffc1ba2547149c1889158e1650f6, State: Initialized, Role: FOLLOWER
I20260812 01:19:18.426533 378 consensus_queue.cc:260] T f5df26fb669845569eab70987d6af066 P a31cffc1ba2547149c1889158e1650f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a31cffc1ba2547149c1889158e1650f6" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 36041 } } peers { permanent_uuid: "9d5ed02e09f64c00bba7c9e0c57a9f63" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 45861 } } peers { permanent_uuid: "3a24ca4abd994042a92e9b1d2577152a" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 40381 } }
I20260812 01:19:18.426631 384 tablet_bootstrap.cc:492] T f5df26fb669845569eab70987d6af066 P 9d5ed02e09f64c00bba7c9e0c57a9f63: No bootstrap required, opened a new log
I20260812 01:19:18.426700 384 ts_tablet_manager.cc:1397] T f5df26fb669845569eab70987d6af066 P 9d5ed02e09f64c00bba7c9e0c57a9f63: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 01:19:18.426839 378 ts_tablet_manager.cc:1428] T f5df26fb669845569eab70987d6af066 P a31cffc1ba2547149c1889158e1650f6: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 01:19:18.426764 375 raft_consensus.cc:348] T f5df26fb669845569eab70987d6af066 P 3a24ca4abd994042a92e9b1d2577152a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a31cffc1ba2547149c1889158e1650f6" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 36041 } } peers { permanent_uuid: "9d5ed02e09f64c00bba7c9e0c57a9f63" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 45861 } } peers { permanent_uuid: "3a24ca4abd994042a92e9b1d2577152a" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 40381 } }
I20260812 01:19:18.426906 375 raft_consensus.cc:374] T f5df26fb669845569eab70987d6af066 P 3a24ca4abd994042a92e9b1d2577152a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:19:18.426951 375 raft_consensus.cc:729] T f5df26fb669845569eab70987d6af066 P 3a24ca4abd994042a92e9b1d2577152a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a24ca4abd994042a92e9b1d2577152a, State: Initialized, Role: FOLLOWER
I20260812 01:19:18.426975 354 heartbeater.cc:503] Master 127.24.0.190:33889 was elected leader, sending a full tablet report...
I20260812 01:19:18.427068 375 consensus_queue.cc:260] T f5df26fb669845569eab70987d6af066 P 3a24ca4abd994042a92e9b1d2577152a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a31cffc1ba2547149c1889158e1650f6" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 36041 } } peers { permanent_uuid: "9d5ed02e09f64c00bba7c9e0c57a9f63" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 45861 } } peers { permanent_uuid: "3a24ca4abd994042a92e9b1d2577152a" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 40381 } }
I20260812 01:19:18.427361 375 ts_tablet_manager.cc:1428] T f5df26fb669845569eab70987d6af066 P 3a24ca4abd994042a92e9b1d2577152a: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
I20260812 01:19:18.427472 32405 heartbeater.cc:503] Master 127.24.0.190:33889 was elected leader, sending a full tablet report...
I20260812 01:19:18.429279 384 raft_consensus.cc:348] T f5df26fb669845569eab70987d6af066 P 9d5ed02e09f64c00bba7c9e0c57a9f63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a31cffc1ba2547149c1889158e1650f6" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 36041 } } peers { permanent_uuid: "9d5ed02e09f64c00bba7c9e0c57a9f63" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 45861 } } peers { permanent_uuid: "3a24ca4abd994042a92e9b1d2577152a" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 40381 } }
I20260812 01:19:18.429392 384 raft_consensus.cc:374] T f5df26fb669845569eab70987d6af066 P 9d5ed02e09f64c00bba7c9e0c57a9f63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:19:18.429414 384 raft_consensus.cc:729] T f5df26fb669845569eab70987d6af066 P 9d5ed02e09f64c00bba7c9e0c57a9f63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d5ed02e09f64c00bba7c9e0c57a9f63, State: Initialized, Role: FOLLOWER
I20260812 01:19:18.429517 384 consensus_queue.cc:260] T f5df26fb669845569eab70987d6af066 P 9d5ed02e09f64c00bba7c9e0c57a9f63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a31cffc1ba2547149c1889158e1650f6" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 36041 } } peers { permanent_uuid: "9d5ed02e09f64c00bba7c9e0c57a9f63" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 45861 } } peers { permanent_uuid: "3a24ca4abd994042a92e9b1d2577152a" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 40381 } }
I20260812 01:19:18.429780 384 ts_tablet_manager.cc:1428] T f5df26fb669845569eab70987d6af066 P 9d5ed02e09f64c00bba7c9e0c57a9f63: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 01:19:18.429811 32615 heartbeater.cc:503] Master 127.24.0.190:33889 was elected leader, sending a full tablet report...
W20260812 01:19:18.536298 32617 tablet.cc:2369] T f5df26fb669845569eab70987d6af066 P 9d5ed02e09f64c00bba7c9e0c57a9f63: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 01:19:18.607329 390 raft_consensus.cc:482] T f5df26fb669845569eab70987d6af066 P 9d5ed02e09f64c00bba7c9e0c57a9f63 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 01:19:18.607499 390 raft_consensus.cc:504] T f5df26fb669845569eab70987d6af066 P 9d5ed02e09f64c00bba7c9e0c57a9f63 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a31cffc1ba2547149c1889158e1650f6" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 36041 } } peers { permanent_uuid: "9d5ed02e09f64c00bba7c9e0c57a9f63" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 45861 } } peers { permanent_uuid: "3a24ca4abd994042a92e9b1d2577152a" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 40381 } }
I20260812 01:19:18.608003 390 leader_election.cc:288] T f5df26fb669845569eab70987d6af066 P 9d5ed02e09f64c00bba7c9e0c57a9f63 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a31cffc1ba2547149c1889158e1650f6 (127.24.0.131:36041), 3a24ca4abd994042a92e9b1d2577152a (127.24.0.129:40381)
I20260812 01:19:18.609418 32619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:18.608250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.129:40381 (local address 127.24.0.130:55469)
0812 01:19:18.608299 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:18.608311 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:18.608355 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:18.608496 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:18.608497 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:18.608500 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:18.609043 (+ 543us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 01:19:18.609043 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:18.609052 (+ 9us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 01:19:18.609067 (+ 15us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:19:18.609255 (+ 188us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:19:18.609257 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:19:18.609257 (+ 0us) client_negotiation.cc:769] Sending connection context
0812 01:19:18.609331 (+ 74us) client_negotiation.cc:240] Negotiation successful
0812 01:19:18.609348 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260812 01:19:18.609418 370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:18.608310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.130:55469 (local address 127.24.0.129:40381)
0812 01:19:18.608363 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:18.608365 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:18.608377 (+ 12us) server_negotiation.cc:407] Connection header received
0812 01:19:18.608409 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:18.608410 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:18.608416 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:18.608471 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:18.609157 (+ 686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:18.609159 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:18.609159 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:19:18.609163 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:18.609180 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:19:18.609185 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:18.609251 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:18.609337 (+ 86us) server_negotiation.cc:299] Negotiation successful
0812 01:19:18.609352 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 01:19:18.609531 371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:18.608466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.130:40135 (local address 127.24.0.131:36041)
0812 01:19:18.608513 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:18.608515 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:18.608528 (+ 13us) server_negotiation.cc:407] Connection header received
0812 01:19:18.608620 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:18.608621 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:18.608625 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:18.608714 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:18.609312 (+ 598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:18.609313 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:18.609314 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:19:18.609317 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:18.609333 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:19:18.609337 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:18.609389 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:18.609452 (+ 63us) server_negotiation.cc:299] Negotiation successful
0812 01:19:18.609466 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 01:19:18.609545 391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:18.608307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.131:36041 (local address 127.24.0.130:40135)
0812 01:19:18.608476 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:18.608487 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:18.608537 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:18.608724 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:18.608725 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:18.608743 (+ 18us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:18.609220 (+ 477us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 01:19:18.609221 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:18.609228 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 01:19:18.609241 (+ 13us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:19:18.609395 (+ 154us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:19:18.609396 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:19:18.609397 (+ 1us) client_negotiation.cc:769] Sending connection context
0812 01:19:18.609451 (+ 54us) client_negotiation.cc:240] Negotiation successful
0812 01:19:18.609465 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":115,"threads_started":1}
I20260812 01:19:18.609927 32747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5df26fb669845569eab70987d6af066" candidate_uuid: "9d5ed02e09f64c00bba7c9e0c57a9f63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a31cffc1ba2547149c1889158e1650f6" is_pre_election: true
I20260812 01:19:18.609948 32338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5df26fb669845569eab70987d6af066" candidate_uuid: "9d5ed02e09f64c00bba7c9e0c57a9f63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a24ca4abd994042a92e9b1d2577152a" is_pre_election: true
I20260812 01:19:18.610085 32338 raft_consensus.cc:2454] T f5df26fb669845569eab70987d6af066 P 3a24ca4abd994042a92e9b1d2577152a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d5ed02e09f64c00bba7c9e0c57a9f63 in term 0.
I20260812 01:19:18.610085 32747 raft_consensus.cc:2454] T f5df26fb669845569eab70987d6af066 P a31cffc1ba2547149c1889158e1650f6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d5ed02e09f64c00bba7c9e0c57a9f63 in term 0.
I20260812 01:19:18.610329 32449 leader_election.cc:302] T f5df26fb669845569eab70987d6af066 P 9d5ed02e09f64c00bba7c9e0c57a9f63 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d5ed02e09f64c00bba7c9e0c57a9f63, a31cffc1ba2547149c1889158e1650f6; no voters:
I20260812 01:19:18.610505 390 raft_consensus.cc:2781] T f5df26fb669845569eab70987d6af066 P 9d5ed02e09f64c00bba7c9e0c57a9f63 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 01:19:18.610563 390 raft_consensus.cc:482] T f5df26fb669845569eab70987d6af066 P 9d5ed02e09f64c00bba7c9e0c57a9f63 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 01:19:18.610591 390 raft_consensus.cc:3037] T f5df26fb669845569eab70987d6af066 P 9d5ed02e09f64c00bba7c9e0c57a9f63 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:19:18.611308 390 raft_consensus.cc:504] T f5df26fb669845569eab70987d6af066 P 9d5ed02e09f64c00bba7c9e0c57a9f63 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a31cffc1ba2547149c1889158e1650f6" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 36041 } } peers { permanent_uuid: "9d5ed02e09f64c00bba7c9e0c57a9f63" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 45861 } } peers { permanent_uuid: "3a24ca4abd994042a92e9b1d2577152a" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 40381 } }
I20260812 01:19:18.611449 390 leader_election.cc:288] T f5df26fb669845569eab70987d6af066 P 9d5ed02e09f64c00bba7c9e0c57a9f63 [CANDIDATE]: Term 1 election: Requested vote from peers a31cffc1ba2547149c1889158e1650f6 (127.24.0.131:36041), 3a24ca4abd994042a92e9b1d2577152a (127.24.0.129:40381)
I20260812 01:19:18.611581 32747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5df26fb669845569eab70987d6af066" candidate_uuid: "9d5ed02e09f64c00bba7c9e0c57a9f63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a31cffc1ba2547149c1889158e1650f6"
I20260812 01:19:18.611594 32338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5df26fb669845569eab70987d6af066" candidate_uuid: "9d5ed02e09f64c00bba7c9e0c57a9f63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a24ca4abd994042a92e9b1d2577152a"
I20260812 01:19:18.611665 32747 raft_consensus.cc:3037] T f5df26fb669845569eab70987d6af066 P a31cffc1ba2547149c1889158e1650f6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:19:18.611673 32338 raft_consensus.cc:3037] T f5df26fb669845569eab70987d6af066 P 3a24ca4abd994042a92e9b1d2577152a [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:19:18.612613 32338 raft_consensus.cc:2454] T f5df26fb669845569eab70987d6af066 P 3a24ca4abd994042a92e9b1d2577152a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d5ed02e09f64c00bba7c9e0c57a9f63 in term 1.
I20260812 01:19:18.612612 32747 raft_consensus.cc:2454] T f5df26fb669845569eab70987d6af066 P a31cffc1ba2547149c1889158e1650f6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d5ed02e09f64c00bba7c9e0c57a9f63 in term 1.
I20260812 01:19:18.612811 32445 leader_election.cc:302] T f5df26fb669845569eab70987d6af066 P 9d5ed02e09f64c00bba7c9e0c57a9f63 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a24ca4abd994042a92e9b1d2577152a, 9d5ed02e09f64c00bba7c9e0c57a9f63; no voters:
I20260812 01:19:18.612914 390 raft_consensus.cc:2781] T f5df26fb669845569eab70987d6af066 P 9d5ed02e09f64c00bba7c9e0c57a9f63 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:19:18.613060 390 raft_consensus.cc:686] T f5df26fb669845569eab70987d6af066 P 9d5ed02e09f64c00bba7c9e0c57a9f63 [term 1 LEADER]: Becoming Leader. State: Replica: 9d5ed02e09f64c00bba7c9e0c57a9f63, State: Running, Role: LEADER
I20260812 01:19:18.613157 390 consensus_queue.cc:237] T f5df26fb669845569eab70987d6af066 P 9d5ed02e09f64c00bba7c9e0c57a9f63 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a31cffc1ba2547149c1889158e1650f6" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 36041 } } peers { permanent_uuid: "9d5ed02e09f64c00bba7c9e0c57a9f63" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 45861 } } peers { permanent_uuid: "3a24ca4abd994042a92e9b1d2577152a" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 40381 } }
I20260812 01:19:18.613998 32105 catalog_manager.cc:5495] T f5df26fb669845569eab70987d6af066 P 9d5ed02e09f64c00bba7c9e0c57a9f63 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9d5ed02e09f64c00bba7c9e0c57a9f63 (127.24.0.130). New cstate: current_term: 1 leader_uuid: "9d5ed02e09f64c00bba7c9e0c57a9f63" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a31cffc1ba2547149c1889158e1650f6" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 36041 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d5ed02e09f64c00bba7c9e0c57a9f63" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 45861 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a24ca4abd994042a92e9b1d2577152a" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 40381 } health_report { overall_health: UNKNOWN } } }
I20260812 01:19:18.646919 374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:18.645895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47826 (local address 127.24.0.130:45861)
0812 01:19:18.645955 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:18.645959 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:18.645972 (+ 13us) server_negotiation.cc:407] Connection header received
0812 01:19:18.646013 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:18.646014 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:18.646028 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:18.646088 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:18.646655 (+ 567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:18.646657 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:18.646657 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:19:18.646662 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:18.646677 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:19:18.646681 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:18.646731 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:18.646801 (+ 70us) server_negotiation.cc:299] Negotiation successful
0812 01:19:18.646813 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260812 01:19:18.649106 32747 raft_consensus.cc:1261] T f5df26fb669845569eab70987d6af066 P a31cffc1ba2547149c1889158e1650f6 [term 1 FOLLOWER]: Refusing update from remote peer 9d5ed02e09f64c00bba7c9e0c57a9f63: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:19:18.649097 32338 raft_consensus.cc:1261] T f5df26fb669845569eab70987d6af066 P 3a24ca4abd994042a92e9b1d2577152a [term 1 FOLLOWER]: Refusing update from remote peer 9d5ed02e09f64c00bba7c9e0c57a9f63: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:19:18.649580 390 consensus_queue.cc:1035] T f5df26fb669845569eab70987d6af066 P 9d5ed02e09f64c00bba7c9e0c57a9f63 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a24ca4abd994042a92e9b1d2577152a" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 40381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:19:18.649765 392 consensus_queue.cc:1035] T f5df26fb669845569eab70987d6af066 P 9d5ed02e09f64c00bba7c9e0c57a9f63 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a31cffc1ba2547149c1889158e1650f6" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 36041 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 01:19:18.652168 355 tablet.cc:2369] T f5df26fb669845569eab70987d6af066 P a31cffc1ba2547149c1889158e1650f6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 01:19:18.653535 403 mvcc.cc:205] Tried to move back new op lower bound from 7317494000223338496 to 7317494000080048128. Current Snapshot: MvccSnapshot[applied={T|T < 7317494000223338496 or (T in {7317494000223338496})}]
I20260812 01:19:18.655097 32105 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34548:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:19:18.655205 32105 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34548:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:19:18.656098 32105 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e31aba4353e746e4bb67c46bfc6e9e59: Sending DeleteTablet for 3 replicas of tablet f5df26fb669845569eab70987d6af066
I20260812 01:19:18.656497 32716 tablet_service.cc:1514] Processing DeleteTablet for tablet f5df26fb669845569eab70987d6af066 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:19:18 UTC) from {username='slave'} at 127.0.0.1:56882
I20260812 01:19:18.656759 424 tablet_replica.cc:330] T f5df26fb669845569eab70987d6af066 P a31cffc1ba2547149c1889158e1650f6: stopping tablet replica
I20260812 01:19:18.656863 424 raft_consensus.cc:2229] T f5df26fb669845569eab70987d6af066 P a31cffc1ba2547149c1889158e1650f6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 01:19:18.656877 32514 tablet_service.cc:1514] Processing DeleteTablet for tablet f5df26fb669845569eab70987d6af066 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:19:18 UTC) from {username='slave'} at 127.0.0.1:47822
I20260812 01:19:18.656970 424 raft_consensus.cc:2258] T f5df26fb669845569eab70987d6af066 P a31cffc1ba2547149c1889158e1650f6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:19:18.657222 32304 tablet_service.cc:1514] Processing DeleteTablet for tablet f5df26fb669845569eab70987d6af066 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:19:18 UTC) from {username='slave'} at 127.0.0.1:54380
I20260812 01:19:18.657526 424 ts_tablet_manager.cc:1905] T f5df26fb669845569eab70987d6af066 P a31cffc1ba2547149c1889158e1650f6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:19:18.657809 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 32192
I20260812 01:19:18.667749 424 ts_tablet_manager.cc:1918] T f5df26fb669845569eab70987d6af066 P a31cffc1ba2547149c1889158e1650f6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 01:19:18.667858 424 log.cc:1196] T f5df26fb669845569eab70987d6af066 P a31cffc1ba2547149c1889158e1650f6: Deleting WAL directory at /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786497534195564-24578-0/minicluster-data/ts-2/wal/wals/f5df26fb669845569eab70987d6af066
I20260812 01:19:18.668180 424 ts_tablet_manager.cc:1939] T f5df26fb669845569eab70987d6af066 P a31cffc1ba2547149c1889158e1650f6: Deleting consensus metadata
I20260812 01:19:18.668731 32086 catalog_manager.cc:4841] TS a31cffc1ba2547149c1889158e1650f6 (127.24.0.131:36041): tablet f5df26fb669845569eab70987d6af066 (table test-table [id=13286eb0a16a41239f40920b13569fbc]) successfully deleted
I20260812 01:19:18.670913 426 tablet_replica.cc:330] T f5df26fb669845569eab70987d6af066 P 9d5ed02e09f64c00bba7c9e0c57a9f63: stopping tablet replica
I20260812 01:19:18.671056 426 raft_consensus.cc:2229] T f5df26fb669845569eab70987d6af066 P 9d5ed02e09f64c00bba7c9e0c57a9f63 [term 1 LEADER]: Raft consensus shutting down.
I20260812 01:19:18.671170 426 raft_consensus.cc:2258] T f5df26fb669845569eab70987d6af066 P 9d5ed02e09f64c00bba7c9e0c57a9f63 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:19:18.671864 426 ts_tablet_manager.cc:1905] T f5df26fb669845569eab70987d6af066 P 9d5ed02e09f64c00bba7c9e0c57a9f63: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260812 01:19:18.673799 32084 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.0.129:40381 (error 108)
I20260812 01:19:18.674003 426 ts_tablet_manager.cc:1918] T f5df26fb669845569eab70987d6af066 P 9d5ed02e09f64c00bba7c9e0c57a9f63: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 01:19:18.674160 426 log.cc:1196] T f5df26fb669845569eab70987d6af066 P 9d5ed02e09f64c00bba7c9e0c57a9f63: Deleting WAL directory at /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786497534195564-24578-0/minicluster-data/ts-1/wal/wals/f5df26fb669845569eab70987d6af066
I20260812 01:19:18.674394 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 32414
I20260812 01:19:18.674598 426 ts_tablet_manager.cc:1939] T f5df26fb669845569eab70987d6af066 P 9d5ed02e09f64c00bba7c9e0c57a9f63: Deleting consensus metadata
I20260812 01:19:18.677040 368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:18.676876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.129:40381 (local address 127.0.0.1:54392)
0812 01:19:18.676942 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:18.676990 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.129:40381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260812 01:19:18.677268 32084 catalog_manager.cc:4584] TS 3a24ca4abd994042a92e9b1d2577152a (127.24.0.129:40381): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f5df26fb669845569eab70987d6af066: Network error: Client connection negotiation failed: client connection to 127.24.0.129:40381: connect: Connection refused (error 111)
I20260812 01:19:18.685063 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 32621
I20260812 01:19:18.685279 368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:18.685088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.130:45861 (local address 127.0.0.1:47836)
0812 01:19:18.685203 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:18.685238 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.130:45861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260812 01:19:18.693022 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 32057
2026-08-12T01:19:18Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (834 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260812 01:19:18.715849 24578 mini_oidc.cc:152] Starting JWKS server
I20260812 01:19:18.717016 24578 webserver.cc:460] Webserver started at https://127.0.0.1:41661/ using document root <none> and password file <none>
I20260812 01:19:18.717139 24578 mini_oidc.cc:189] Starting OIDC Discovery server
I20260812 01:19:18.717330 24578 webserver.cc:460] Webserver started at http://127.0.0.1:33469/ using document root <none> and password file <none>
2026-08-12T01:19:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:19:18Z Disabled control of system clock
I20260812 01:19:18.726292 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786497534195564-24578-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786497534195564-24578-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786497534195564-24578-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.190:37191
--webserver_interface=127.24.0.190
--webserver_port=0
--builtin_ntp_servers=127.24.0.148:34531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.0.190:37191
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41661/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786497534195564-24578-0/testchainca.cert with env {}
W20260812 01:19:18.809969 440 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:19:18.810182 440 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:19:18.810204 440 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:19:18.811357 440 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:19:18.811395 440 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:19:18.811408 440 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:19:18.811420 440 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:19:18.811432 440 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 01:19:18.811489 440 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 01:19:18.811506 440 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:41661/jwks.json
I20260812 01:19:18.813072 440 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:34531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786497534195564-24578-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786497534195564-24578-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.0.190:37191
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.190:37191
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41661/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.0.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786497534195564-24578-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.440
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786497534195564-24578-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:19:18.813336 440 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:19:18.813561 440 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:19:18.818354 447 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:18.818380 445 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:18.818380 450 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:19:18.818629 440 server_base.cc:1034] running on GCE node
I20260812 01:19:18.818922 440 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:19:18.819212 440 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:19:18.820358 440 hybrid_clock.cc:648] HybridClock initialized: now 1786497558820337 us; error 38 us; skew 500 ppm
I20260812 01:19:18.821679 440 webserver.cc:460] Webserver started at http://127.24.0.190:32917/ using document root <none> and password file <none>
I20260812 01:19:18.821925 440 fs_manager.cc:359] Metadata directory not provided
I20260812 01:19:18.821973 440 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:19:18.822151 440 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:19:18.823181 440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786497534195564-24578-0/minicluster-data/master-0/data/instance:
uuid: "b066064cb3244e34b8ad28c41a8c82b5"
format_stamp: "Formatted at 2026-08-12 01:19:18 on dist-test-slave-t3q3"
server_key: "cca334c427ea830460bd5ce40c209445"
server_key_iv: "7ae00361a95f21c818cdeab98a216d94"
server_key_version: "encryptionkey@0"
I20260812 01:19:18.823559 440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786497534195564-24578-0/minicluster-data/master-0/wal/instance:
uuid: "b066064cb3244e34b8ad28c41a8c82b5"
format_stamp: "Formatted at 2026-08-12 01:19:18 on dist-test-slave-t3q3"
server_key: "cca334c427ea830460bd5ce40c209445"
server_key_iv: "7ae00361a95f21c818cdeab98a216d94"
server_key_version: "encryptionkey@0"
I20260812 01:19:18.824944 440 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 01:19:18.825692 458 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:18.825873 440 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 01:19:18.825980 440 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786497534195564-24578-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786497534195564-24578-0/minicluster-data/master-0/wal
uuid: "b066064cb3244e34b8ad28c41a8c82b5"
format_stamp: "Formatted at 2026-08-12 01:19:18 on dist-test-slave-t3q3"
server_key: "cca334c427ea830460bd5ce40c209445"
server_key_iv: "7ae00361a95f21c818cdeab98a216d94"
server_key_version: "encryptionkey@0"
I20260812 01:19:18.826056 440 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786497534195564-24578-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786497534195564-24578-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786497534195564-24578-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:19:18.838992 440 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:19:18.839331 440 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:19:18.839494 440 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:19:18.843580 440 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.190:37191
I20260812 01:19:18.843611 531 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.190:37191 every 8 connection(s)
I20260812 01:19:18.843981 440 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
I20260812 01:19:18.845158 532 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:19:18.847592 532 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b066064cb3244e34b8ad28c41a8c82b5: Bootstrap starting.
I20260812 01:19:18.848269 532 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b066064cb3244e34b8ad28c41a8c82b5: Neither blocks nor log segments found. Creating new log.
I20260812 01:19:18.848594 532 log.cc:824] T 00000000000000000000000000000000 P b066064cb3244e34b8ad28c41a8c82b5: Log is configured to *not* fsync() on all Append() calls
I20260812 01:19:18.849328 532 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b066064cb3244e34b8ad28c41a8c82b5: No bootstrap required, opened a new log
I20260812 01:19:18.850930 532 raft_consensus.cc:348] T 00000000000000000000000000000000 P b066064cb3244e34b8ad28c41a8c82b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b066064cb3244e34b8ad28c41a8c82b5" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 37191 } }
I20260812 01:19:18.851155 532 raft_consensus.cc:374] T 00000000000000000000000000000000 P b066064cb3244e34b8ad28c41a8c82b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:19:18.851198 532 raft_consensus.cc:729] T 00000000000000000000000000000000 P b066064cb3244e34b8ad28c41a8c82b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b066064cb3244e34b8ad28c41a8c82b5, State: Initialized, Role: FOLLOWER
I20260812 01:19:18.851207 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 440
I20260812 01:19:18.851316 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786497534195564-24578-0/minicluster-data/master-0/wal/instance
I20260812 01:19:18.851367 532 consensus_queue.cc:260] T 00000000000000000000000000000000 P b066064cb3244e34b8ad28c41a8c82b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b066064cb3244e34b8ad28c41a8c82b5" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 37191 } }
I20260812 01:19:18.851456 532 raft_consensus.cc:388] T 00000000000000000000000000000000 P b066064cb3244e34b8ad28c41a8c82b5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:19:18.851498 532 raft_consensus.cc:482] T 00000000000000000000000000000000 P b066064cb3244e34b8ad28c41a8c82b5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:19:18.851519 24578 external_mini_cluster.cc:1428] Setting key e6891eee0dc0a92e4a9776ce260abe6f
I20260812 01:19:18.851552 532 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b066064cb3244e34b8ad28c41a8c82b5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:19:18.852311 532 raft_consensus.cc:504] T 00000000000000000000000000000000 P b066064cb3244e34b8ad28c41a8c82b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b066064cb3244e34b8ad28c41a8c82b5" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 37191 } }
I20260812 01:19:18.852407 532 leader_election.cc:302] T 00000000000000000000000000000000 P b066064cb3244e34b8ad28c41a8c82b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b066064cb3244e34b8ad28c41a8c82b5; no voters:
I20260812 01:19:18.852598 532 leader_election.cc:288] T 00000000000000000000000000000000 P b066064cb3244e34b8ad28c41a8c82b5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 01:19:18.852627 547 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b066064cb3244e34b8ad28c41a8c82b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:19:18.852735 547 raft_consensus.cc:686] T 00000000000000000000000000000000 P b066064cb3244e34b8ad28c41a8c82b5 [term 1 LEADER]: Becoming Leader. State: Replica: b066064cb3244e34b8ad28c41a8c82b5, State: Running, Role: LEADER
I20260812 01:19:18.852815 547 consensus_queue.cc:237] T 00000000000000000000000000000000 P b066064cb3244e34b8ad28c41a8c82b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b066064cb3244e34b8ad28c41a8c82b5" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 37191 } }
I20260812 01:19:18.852861 532 sys_catalog.cc:564] T 00000000000000000000000000000000 P b066064cb3244e34b8ad28c41a8c82b5 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:19:18.853147 547 sys_catalog.cc:455] T 00000000000000000000000000000000 P b066064cb3244e34b8ad28c41a8c82b5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b066064cb3244e34b8ad28c41a8c82b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b066064cb3244e34b8ad28c41a8c82b5" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 37191 } } }
I20260812 01:19:18.853220 547 sys_catalog.cc:458] T 00000000000000000000000000000000 P b066064cb3244e34b8ad28c41a8c82b5 [sys.catalog]: This master's current role is: LEADER
I20260812 01:19:18.853595 549 sys_catalog.cc:455] T 00000000000000000000000000000000 P b066064cb3244e34b8ad28c41a8c82b5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b066064cb3244e34b8ad28c41a8c82b5. Latest consensus state: current_term: 1 leader_uuid: "b066064cb3244e34b8ad28c41a8c82b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b066064cb3244e34b8ad28c41a8c82b5" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 37191 } } }
I20260812 01:19:18.853686 553 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:19:18.853703 549 sys_catalog.cc:458] T 00000000000000000000000000000000 P b066064cb3244e34b8ad28c41a8c82b5 [sys.catalog]: This master's current role is: LEADER
I20260812 01:19:18.854287 553 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:19:18.855975 553 catalog_manager.cc:1347] Generated new cluster ID: 610371a9d56e4c8e95464493428fb0a0
I20260812 01:19:18.856035 553 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 01:19:18.858312 542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:18.852100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49106 (local address 127.24.0.190:37191)
0812 01:19:18.852341 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:18.852349 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:18.852398 (+ 49us) server_negotiation.cc:407] Connection header received
0812 01:19:18.852540 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:18.852546 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:18.852618 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:18.852778 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:18.853073 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:18.855326 (+ 2253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:18.856337 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:18.856759 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:18.856909 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:18.857653 (+ 744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:18.857657 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:18.857657 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:19:18.857843 (+ 186us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:18.857876 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:19:18.857887 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:18.857963 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:18.858042 (+ 79us) server_negotiation.cc:299] Negotiation successful
0812 01:19:18.858122 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":125,"threads_started":1}
I20260812 01:19:18.859117 24578 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 01:19:18.868124 553 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:19:18.868626 553 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:19:18.873214 553 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b066064cb3244e34b8ad28c41a8c82b5: Generated new TSK 0
I20260812 01:19:18.873389 553 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260812 01:19:18.905604 584 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260812 01:19:18.905776 584 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260812 01:19:18.906157 542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:18.887882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49116 (local address 127.24.0.190:37191)
0812 01:19:18.887950 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:18.887953 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:18.888211 (+ 258us) server_negotiation.cc:407] Connection header received
0812 01:19:18.888294 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:18.888295 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:18.888309 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:18.888394 (+ 85us) server_negotiation.cc:226] Negotiated authn=JWT
0812 01:19:18.888549 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:18.889344 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:18.890127 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:18.890436 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:18.890507 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:18.890709 (+ 202us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 01:19:18.905692 (+ 14983us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41661/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0812 01:19:18.906105 (+ 413us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:49116: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41661/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":39}
W20260812 01:19:18.906229 542 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:49116: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41661/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260812 01:19:18.906395 582 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:19:18.887826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:37191 (local address 127.0.0.1:49116)
0812 01:19:18.888137 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:18.888147 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:18.888211 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:18.888410 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:18.888410 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:18.888416 (+ 6us) client_negotiation.cc:189] Negotiated authn=JWT
0812 01:19:18.888474 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:18.888476 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:18.889436 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:18.889437 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:18.890042 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:18.890044 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:18.890532 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:18.890533 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:18.890578 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:18.890597 (+ 19us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 01:19:18.905859 (+ 15262us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41661/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0812 01:19:18.905969 (+ 110us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.24.0.190:37191: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41661/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":126,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.24.0.190:37191: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41661/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260812 01:19:18.907012 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 440
2026-08-12T01:19:18Z chronyd exiting
I20260812 01:19:18.950866 24578 test_util.cc:175] -----------------------------------------------
I20260812 01:19:18.950953 24578 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786497534195564-24578-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (236 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260812 01:19:18.951723 24578 mini_oidc.cc:152] Starting JWKS server
I20260812 01:19:18.952445 24578 webserver.cc:460] Webserver started at https://127.0.0.1:40377/ using document root <none> and password file <none>
I20260812 01:19:18.952515 24578 mini_oidc.cc:189] Starting OIDC Discovery server
I20260812 01:19:18.952715 24578 webserver.cc:460] Webserver started at http://127.0.0.1:37897/ using document root <none> and password file <none>
2026-08-12T01:19:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:19:18Z Disabled control of system clock
I20260812 01:19:18.962028 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786497534195564-24578-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786497534195564-24578-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786497534195564-24578-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.190:36405
--webserver_interface=127.24.0.190
--webserver_port=0
--builtin_ntp_servers=127.24.0.148:39039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.0.190:36405
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40377/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786497534195564-24578-0/testchainca.cert with env {}
W20260812 01:19:19.043584 592 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:19:19.043785 592 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:19:19.043807 592 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:19:19.045001 592 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:19:19.045045 592 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:19:19.045058 592 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:19:19.045107 592 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:19:19.045137 592 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 01:19:19.045161 592 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 01:19:19.045185 592 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:40377/jwks.json
I20260812 01:19:19.046713 592 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:39039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786497534195564-24578-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786497534195564-24578-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.0.190:36405
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.190:36405
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40377/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.0.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786497534195564-24578-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.592
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786497534195564-24578-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:19:19.046983 592 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:19:19.047209 592 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:19:19.052115 604 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:19.052134 599 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:19.052181 602 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:19:19.052366 592 server_base.cc:1034] running on GCE node
I20260812 01:19:19.052644 592 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:19:19.052938 592 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:19:19.054111 592 hybrid_clock.cc:648] HybridClock initialized: now 1786497559054023 us; error 104 us; skew 500 ppm
I20260812 01:19:19.055455 592 webserver.cc:460] Webserver started at http://127.24.0.190:35369/ using document root <none> and password file <none>
I20260812 01:19:19.055735 592 fs_manager.cc:359] Metadata directory not provided
I20260812 01:19:19.055784 592 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:19:19.055958 592 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:19:19.057089 592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786497534195564-24578-0/minicluster-data/master-0/data/instance:
uuid: "844a0de6aa7c426ea6d9514abcae62fc"
format_stamp: "Formatted at 2026-08-12 01:19:19 on dist-test-slave-t3q3"
server_key: "134ba0fdccafb2550dc6cf3c1382cef1"
server_key_iv: "593e1a8e7c14b5294134a8f4d74e1635"
server_key_version: "encryptionkey@0"
I20260812 01:19:19.057507 592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786497534195564-24578-0/minicluster-data/master-0/wal/instance:
uuid: "844a0de6aa7c426ea6d9514abcae62fc"
format_stamp: "Formatted at 2026-08-12 01:19:19 on dist-test-slave-t3q3"
server_key: "134ba0fdccafb2550dc6cf3c1382cef1"
server_key_iv: "593e1a8e7c14b5294134a8f4d74e1635"
server_key_version: "encryptionkey@0"
I20260812 01:19:19.058903 592 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 01:19:19.059609 614 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:19.059813 592 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 01:19:19.059906 592 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786497534195564-24578-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786497534195564-24578-0/minicluster-data/master-0/wal
uuid: "844a0de6aa7c426ea6d9514abcae62fc"
format_stamp: "Formatted at 2026-08-12 01:19:19 on dist-test-slave-t3q3"
server_key: "134ba0fdccafb2550dc6cf3c1382cef1"
server_key_iv: "593e1a8e7c14b5294134a8f4d74e1635"
server_key_version: "encryptionkey@0"
I20260812 01:19:19.059988 592 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786497534195564-24578-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786497534195564-24578-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786497534195564-24578-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:19:19.077701 592 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:19:19.078040 592 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:19:19.078208 592 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:19:19.082491 592 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.190:36405
I20260812 01:19:19.082531 697 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.190:36405 every 8 connection(s)
I20260812 01:19:19.082927 592 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
I20260812 01:19:19.083645 698 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:19:19.086495 698 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 844a0de6aa7c426ea6d9514abcae62fc: Bootstrap starting.
I20260812 01:19:19.086741 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 592
I20260812 01:19:19.086859 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786497534195564-24578-0/minicluster-data/master-0/wal/instance
I20260812 01:19:19.087042 24578 external_mini_cluster.cc:1428] Setting key 39618ad7e685987f27ece51639a8e4db
I20260812 01:19:19.087317 698 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 844a0de6aa7c426ea6d9514abcae62fc: Neither blocks nor log segments found. Creating new log.
I20260812 01:19:19.087677 698 log.cc:824] T 00000000000000000000000000000000 P 844a0de6aa7c426ea6d9514abcae62fc: Log is configured to *not* fsync() on all Append() calls
I20260812 01:19:19.088557 698 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 844a0de6aa7c426ea6d9514abcae62fc: No bootstrap required, opened a new log
I20260812 01:19:19.090915 698 raft_consensus.cc:348] T 00000000000000000000000000000000 P 844a0de6aa7c426ea6d9514abcae62fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "844a0de6aa7c426ea6d9514abcae62fc" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 36405 } }
I20260812 01:19:19.091075 698 raft_consensus.cc:374] T 00000000000000000000000000000000 P 844a0de6aa7c426ea6d9514abcae62fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:19:19.091113 698 raft_consensus.cc:729] T 00000000000000000000000000000000 P 844a0de6aa7c426ea6d9514abcae62fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 844a0de6aa7c426ea6d9514abcae62fc, State: Initialized, Role: FOLLOWER
I20260812 01:19:19.091238 698 consensus_queue.cc:260] T 00000000000000000000000000000000 P 844a0de6aa7c426ea6d9514abcae62fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "844a0de6aa7c426ea6d9514abcae62fc" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 36405 } }
I20260812 01:19:19.091310 698 raft_consensus.cc:388] T 00000000000000000000000000000000 P 844a0de6aa7c426ea6d9514abcae62fc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:19:19.091341 698 raft_consensus.cc:482] T 00000000000000000000000000000000 P 844a0de6aa7c426ea6d9514abcae62fc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:19:19.091385 698 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 844a0de6aa7c426ea6d9514abcae62fc [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:19:19.092130 698 raft_consensus.cc:504] T 00000000000000000000000000000000 P 844a0de6aa7c426ea6d9514abcae62fc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "844a0de6aa7c426ea6d9514abcae62fc" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 36405 } }
I20260812 01:19:19.092267 698 leader_election.cc:302] T 00000000000000000000000000000000 P 844a0de6aa7c426ea6d9514abcae62fc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 844a0de6aa7c426ea6d9514abcae62fc; no voters:
I20260812 01:19:19.092434 698 leader_election.cc:288] T 00000000000000000000000000000000 P 844a0de6aa7c426ea6d9514abcae62fc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 01:19:19.092460 705 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 844a0de6aa7c426ea6d9514abcae62fc [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:19:19.092626 705 raft_consensus.cc:686] T 00000000000000000000000000000000 P 844a0de6aa7c426ea6d9514abcae62fc [term 1 LEADER]: Becoming Leader. State: Replica: 844a0de6aa7c426ea6d9514abcae62fc, State: Running, Role: LEADER
I20260812 01:19:19.092706 698 sys_catalog.cc:564] T 00000000000000000000000000000000 P 844a0de6aa7c426ea6d9514abcae62fc [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:19:19.092720 705 consensus_queue.cc:237] T 00000000000000000000000000000000 P 844a0de6aa7c426ea6d9514abcae62fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "844a0de6aa7c426ea6d9514abcae62fc" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 36405 } }
I20260812 01:19:19.093062 706 sys_catalog.cc:455] T 00000000000000000000000000000000 P 844a0de6aa7c426ea6d9514abcae62fc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "844a0de6aa7c426ea6d9514abcae62fc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "844a0de6aa7c426ea6d9514abcae62fc" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 36405 } } }
I20260812 01:19:19.093045 705 sys_catalog.cc:455] T 00000000000000000000000000000000 P 844a0de6aa7c426ea6d9514abcae62fc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 844a0de6aa7c426ea6d9514abcae62fc. Latest consensus state: current_term: 1 leader_uuid: "844a0de6aa7c426ea6d9514abcae62fc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "844a0de6aa7c426ea6d9514abcae62fc" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 36405 } } }
I20260812 01:19:19.093240 706 sys_catalog.cc:458] T 00000000000000000000000000000000 P 844a0de6aa7c426ea6d9514abcae62fc [sys.catalog]: This master's current role is: LEADER
I20260812 01:19:19.093261 705 sys_catalog.cc:458] T 00000000000000000000000000000000 P 844a0de6aa7c426ea6d9514abcae62fc [sys.catalog]: This master's current role is: LEADER
I20260812 01:19:19.093585 702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:19.087598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52842 (local address 127.24.0.190:36405)
0812 01:19:19.087837 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:19.087846 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:19.087883 (+ 37us) server_negotiation.cc:407] Connection header received
0812 01:19:19.088540 (+ 657us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:19.088548 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:19.088624 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:19.088786 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:19.088966 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:19.090337 (+ 1371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:19.091076 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:19.091701 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:19.091827 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:19.092536 (+ 709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:19.092540 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:19.092541 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:19:19.092762 (+ 221us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:19.092807 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:19:19.092822 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:19.093142 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:19.093175 (+ 33us) server_negotiation.cc:299] Negotiation successful
0812 01:19:19.093259 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":98,"threads_started":1}
I20260812 01:19:19.093641 717 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:19:19.094540 717 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:19:19.094568 24578 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 01:19:19.095984 717 catalog_manager.cc:1347] Generated new cluster ID: 70165283344e4b91bdd07400bbd8f403
I20260812 01:19:19.096035 717 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 01:19:19.108420 717 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:19:19.109126 717 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:19:19.118376 717 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 844a0de6aa7c426ea6d9514abcae62fc: Generated new TSK 0
I20260812 01:19:19.118556 717 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260812 01:19:19.137764 747 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260812 01:19:19.137850 747 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260812 01:19:19.138192 702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:19.123832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52848 (local address 127.24.0.190:36405)
0812 01:19:19.123878 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:19.123880 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:19.123965 (+ 85us) server_negotiation.cc:407] Connection header received
0812 01:19:19.124006 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:19.124007 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:19.124019 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:19.124144 (+ 125us) server_negotiation.cc:226] Negotiated authn=JWT
0812 01:19:19.124278 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:19.125064 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:19.125794 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:19.126102 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:19.126176 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:19.126353 (+ 177us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 01:19:19.137961 (+ 11608us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40377/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0812 01:19:19.138117 (+ 156us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:52848: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40377/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":24}
W20260812 01:19:19.138253 702 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:52848: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40377/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260812 01:19:19.138231 745 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:19:19.123739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:36405 (local address 127.0.0.1:52848)
0812 01:19:19.123883 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:19.123894 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:19.123958 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:19.124146 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:19.124147 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:19.124149 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0812 01:19:19.124208 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:19.124209 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:19.125138 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:19.125139 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:19.125709 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:19.125710 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:19.126191 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:19.126192 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:19.126239 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:19.126244 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 01:19:19.138067 (+ 11823us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40377/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0812 01:19:19.138162 (+ 95us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.24.0.190:36405: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40377/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":97,"threads_started":1}
I20260812 01:19:19.138540 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 592
2026-08-12T01:19:19Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (233 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260812 01:19:19.185781 24578 mini_oidc.cc:152] Starting JWKS server
I20260812 01:19:19.186520 24578 webserver.cc:460] Webserver started at https://127.0.0.1:41659/ using document root <none> and password file <none>
I20260812 01:19:19.186568 24578 mini_oidc.cc:189] Starting OIDC Discovery server
I20260812 01:19:19.186761 24578 webserver.cc:460] Webserver started at http://127.0.0.1:46617/ using document root <none> and password file <none>
2026-08-12T01:19:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:19:19Z Disabled control of system clock
I20260812 01:19:19.196408 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786497534195564-24578-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786497534195564-24578-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786497534195564-24578-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.190:45527
--webserver_interface=127.24.0.190
--webserver_port=0
--builtin_ntp_servers=127.24.0.148:44781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.0.190:45527
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41659/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260812 01:19:19.278563 759 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:19:19.278774 759 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:19:19.278797 759 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:19:19.279884 759 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:19:19.279920 759 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:19:19.279932 759 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:19:19.279944 759 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:19:19.279956 759 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 01:19:19.279968 759 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 01:19:19.279979 759 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:41659/jwks.json
I20260812 01:19:19.281308 759 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:44781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786497534195564-24578-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786497534195564-24578-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.0.190:45527
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.190:45527
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41659/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.0.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.759
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786497534195564-24578-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:19:19.281505 759 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:19:19.281755 759 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:19:19.286849 771 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:19:19.286914 759 server_base.cc:1034] running on GCE node
W20260812 01:19:19.286868 779 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:19.286844 775 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:19:19.287320 759 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:19:19.287628 759 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:19:19.288797 759 hybrid_clock.cc:648] HybridClock initialized: now 1786497559288788 us; error 43 us; skew 500 ppm
I20260812 01:19:19.290172 759 webserver.cc:460] Webserver started at http://127.24.0.190:36733/ using document root <none> and password file <none>
I20260812 01:19:19.290518 759 fs_manager.cc:359] Metadata directory not provided
I20260812 01:19:19.290592 759 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:19:19.290741 759 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:19:19.291832 759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786497534195564-24578-0/minicluster-data/master-0/data/instance:
uuid: "75392a9ba2ad47fd8e9c6df5771a9c07"
format_stamp: "Formatted at 2026-08-12 01:19:19 on dist-test-slave-t3q3"
server_key: "43b0addaf0dc1ea1edb54f7f5523f6a5"
server_key_iv: "9f53d38bd1fe088f3e843240c38e84c8"
server_key_version: "encryptionkey@0"
I20260812 01:19:19.292302 759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786497534195564-24578-0/minicluster-data/master-0/wal/instance:
uuid: "75392a9ba2ad47fd8e9c6df5771a9c07"
format_stamp: "Formatted at 2026-08-12 01:19:19 on dist-test-slave-t3q3"
server_key: "43b0addaf0dc1ea1edb54f7f5523f6a5"
server_key_iv: "9f53d38bd1fe088f3e843240c38e84c8"
server_key_version: "encryptionkey@0"
I20260812 01:19:19.293808 759 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 01:19:19.294545 789 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:19.294723 759 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 01:19:19.294831 759 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786497534195564-24578-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786497534195564-24578-0/minicluster-data/master-0/wal
uuid: "75392a9ba2ad47fd8e9c6df5771a9c07"
format_stamp: "Formatted at 2026-08-12 01:19:19 on dist-test-slave-t3q3"
server_key: "43b0addaf0dc1ea1edb54f7f5523f6a5"
server_key_iv: "9f53d38bd1fe088f3e843240c38e84c8"
server_key_version: "encryptionkey@0"
I20260812 01:19:19.294904 759 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786497534195564-24578-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786497534195564-24578-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786497534195564-24578-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:19:19.325044 759 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:19:19.325433 759 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:19:19.325605 759 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:19:19.329859 759 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.190:45527
I20260812 01:19:19.329929 867 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.190:45527 every 8 connection(s)
I20260812 01:19:19.330308 759 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
I20260812 01:19:19.331022 868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:19:19.331046 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 759
I20260812 01:19:19.331168 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786497534195564-24578-0/minicluster-data/master-0/wal/instance
I20260812 01:19:19.331364 24578 external_mini_cluster.cc:1428] Setting key 699a87f0daf6348bc79f65557f09dc8f
I20260812 01:19:19.335268 868 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75392a9ba2ad47fd8e9c6df5771a9c07: Bootstrap starting.
I20260812 01:19:19.336126 868 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 75392a9ba2ad47fd8e9c6df5771a9c07: Neither blocks nor log segments found. Creating new log.
I20260812 01:19:19.336500 868 log.cc:824] T 00000000000000000000000000000000 P 75392a9ba2ad47fd8e9c6df5771a9c07: Log is configured to *not* fsync() on all Append() calls
I20260812 01:19:19.337393 868 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75392a9ba2ad47fd8e9c6df5771a9c07: No bootstrap required, opened a new log
I20260812 01:19:19.337823 872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:19.331982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53762 (local address 127.24.0.190:45527)
0812 01:19:19.332226 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:19.332235 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:19.332272 (+ 37us) server_negotiation.cc:407] Connection header received
0812 01:19:19.332864 (+ 592us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:19.332874 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:19.332963 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:19.333137 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:19.333364 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:19.334759 (+ 1395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:19.335586 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:19.336273 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:19.336416 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:19.337091 (+ 675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:19.337095 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:19.337095 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:19:19.337311 (+ 216us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:19.337354 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:19:19.337369 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:19.337438 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:19.337562 (+ 124us) server_negotiation.cc:299] Negotiation successful
0812 01:19:19.337660 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":121,"threads_started":1}
I20260812 01:19:19.338901 24578 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 01:19:19.339833 868 raft_consensus.cc:348] T 00000000000000000000000000000000 P 75392a9ba2ad47fd8e9c6df5771a9c07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75392a9ba2ad47fd8e9c6df5771a9c07" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 45527 } }
I20260812 01:19:19.340034 868 raft_consensus.cc:374] T 00000000000000000000000000000000 P 75392a9ba2ad47fd8e9c6df5771a9c07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:19:19.340107 868 raft_consensus.cc:729] T 00000000000000000000000000000000 P 75392a9ba2ad47fd8e9c6df5771a9c07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75392a9ba2ad47fd8e9c6df5771a9c07, State: Initialized, Role: FOLLOWER
I20260812 01:19:19.340240 868 consensus_queue.cc:260] T 00000000000000000000000000000000 P 75392a9ba2ad47fd8e9c6df5771a9c07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75392a9ba2ad47fd8e9c6df5771a9c07" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 45527 } }
I20260812 01:19:19.340340 868 raft_consensus.cc:388] T 00000000000000000000000000000000 P 75392a9ba2ad47fd8e9c6df5771a9c07 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:19:19.340386 868 raft_consensus.cc:482] T 00000000000000000000000000000000 P 75392a9ba2ad47fd8e9c6df5771a9c07 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:19:19.340436 868 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 75392a9ba2ad47fd8e9c6df5771a9c07 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:19:19.340991 868 raft_consensus.cc:504] T 00000000000000000000000000000000 P 75392a9ba2ad47fd8e9c6df5771a9c07 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75392a9ba2ad47fd8e9c6df5771a9c07" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 45527 } }
I20260812 01:19:19.341116 868 leader_election.cc:302] T 00000000000000000000000000000000 P 75392a9ba2ad47fd8e9c6df5771a9c07 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 75392a9ba2ad47fd8e9c6df5771a9c07; no voters:
I20260812 01:19:19.341316 868 leader_election.cc:288] T 00000000000000000000000000000000 P 75392a9ba2ad47fd8e9c6df5771a9c07 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 01:19:19.341356 876 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 75392a9ba2ad47fd8e9c6df5771a9c07 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:19:19.341531 876 raft_consensus.cc:686] T 00000000000000000000000000000000 P 75392a9ba2ad47fd8e9c6df5771a9c07 [term 1 LEADER]: Becoming Leader. State: Replica: 75392a9ba2ad47fd8e9c6df5771a9c07, State: Running, Role: LEADER
I20260812 01:19:19.341583 868 sys_catalog.cc:564] T 00000000000000000000000000000000 P 75392a9ba2ad47fd8e9c6df5771a9c07 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:19:19.341640 876 consensus_queue.cc:237] T 00000000000000000000000000000000 P 75392a9ba2ad47fd8e9c6df5771a9c07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75392a9ba2ad47fd8e9c6df5771a9c07" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 45527 } }
I20260812 01:19:19.342082 879 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75392a9ba2ad47fd8e9c6df5771a9c07 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 75392a9ba2ad47fd8e9c6df5771a9c07. Latest consensus state: current_term: 1 leader_uuid: "75392a9ba2ad47fd8e9c6df5771a9c07" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75392a9ba2ad47fd8e9c6df5771a9c07" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 45527 } } }
I20260812 01:19:19.342051 878 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75392a9ba2ad47fd8e9c6df5771a9c07 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "75392a9ba2ad47fd8e9c6df5771a9c07" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75392a9ba2ad47fd8e9c6df5771a9c07" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 45527 } } }
I20260812 01:19:19.342160 879 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75392a9ba2ad47fd8e9c6df5771a9c07 [sys.catalog]: This master's current role is: LEADER
I20260812 01:19:19.342216 878 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75392a9ba2ad47fd8e9c6df5771a9c07 [sys.catalog]: This master's current role is: LEADER
I20260812 01:19:19.342413 889 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:19:19.342990 889 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:19:19.344668 889 catalog_manager.cc:1347] Generated new cluster ID: 75e4f5b696ea45d88027fb687d695b86
I20260812 01:19:19.344738 889 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 01:19:19.356792 889 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:19:19.357352 889 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:19:19.368634 889 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 75392a9ba2ad47fd8e9c6df5771a9c07: Generated new TSK 0
I20260812 01:19:19.368877 889 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 01:19:19.385357 872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:19.381300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46808 (local address 127.24.0.190:45527)
0812 01:19:19.381364 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:19.381366 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:19.381456 (+ 90us) server_negotiation.cc:407] Connection header received
0812 01:19:19.381517 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:19.381518 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:19.381533 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:19.381624 (+ 91us) server_negotiation.cc:226] Negotiated authn=JWT
0812 01:19:19.381737 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:19.382517 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:19.383333 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:19.383656 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:19.383734 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:19.383901 (+ 167us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 01:19:19.384945 (+ 1044us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41659/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0812 01:19:19.385244 (+ 299us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:46808: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41659/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":37}
W20260812 01:19:19.385450 872 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:46808: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41659/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260812 01:19:19.387203 914 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:19:19.381206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:45527 (local address 127.0.0.1:46808)
0812 01:19:19.381375 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:19.381386 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:19.381456 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:19.381627 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:19.381627 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:19.381629 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0812 01:19:19.381683 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:19.381685 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:19.382647 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:19.382647 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:19.383250 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:19.383252 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:19.383744 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:19.383744 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:19.383787 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:19.383792 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 01:19:19.385058 (+ 1266us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41659/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0812 01:19:19.387099 (+ 2041us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.24.0.190:45527: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41659/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":143,"mutex_wait_us":1814,"thread_start_us":111,"threads_started":1}
I20260812 01:19:19.387598 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 759
E20260812 01:19:19.409198 918 webserver.cc:533] Webserver: sslize failed:
E20260812 01:19:19.409281 918 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-12T01:19:19Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (253 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-12T01:19:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:19:19Z Disabled control of system clock
I20260812 01:19:19.448137 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786497534195564-24578-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786497534195564-24578-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786497534195564-24578-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.190:45555
--webserver_interface=127.24.0.190
--webserver_port=0
--builtin_ntp_servers=127.24.0.148:40883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.0.190:45555
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786497534195564-24578-0/testchainca.cert with env {}
W20260812 01:19:19.530342 930 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:19:19.530707 930 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:19:19.530746 930 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:19:19.532284 930 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:19:19.532362 930 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:19:19.532383 930 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:19:19.532405 930 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:19:19.532433 930 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 01:19:19.532464 930 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 01:19:19.532486 930 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260812 01:19:19.534056 930 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:40883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786497534195564-24578-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786497534195564-24578-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.0.190:45555
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.190:45555
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.0.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786497534195564-24578-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.930
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786497534195564-24578-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:19:19.534359 930 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:19:19.534610 930 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:19:19.539601 938 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:19:19.539757 930 server_base.cc:1034] running on GCE node
W20260812 01:19:19.539613 939 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:19.539609 944 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:19:19.540234 930 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:19:19.540565 930 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:19:19.541743 930 hybrid_clock.cc:648] HybridClock initialized: now 1786497559541731 us; error 42 us; skew 500 ppm
I20260812 01:19:19.543092 930 webserver.cc:460] Webserver started at http://127.24.0.190:45557/ using document root <none> and password file <none>
I20260812 01:19:19.543345 930 fs_manager.cc:359] Metadata directory not provided
I20260812 01:19:19.543416 930 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:19:19.543566 930 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:19:19.544809 930 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786497534195564-24578-0/minicluster-data/master-0/data/instance:
uuid: "4669bcb72fb848c0a3c1d0fbc85976ea"
format_stamp: "Formatted at 2026-08-12 01:19:19 on dist-test-slave-t3q3"
server_key: "ce0182431e7f00a7f4cf661ce570f3fd"
server_key_iv: "bc2606f3b0a4a50366823aa4cffc68d2"
server_key_version: "encryptionkey@0"
I20260812 01:19:19.545235 930 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786497534195564-24578-0/minicluster-data/master-0/wal/instance:
uuid: "4669bcb72fb848c0a3c1d0fbc85976ea"
format_stamp: "Formatted at 2026-08-12 01:19:19 on dist-test-slave-t3q3"
server_key: "ce0182431e7f00a7f4cf661ce570f3fd"
server_key_iv: "bc2606f3b0a4a50366823aa4cffc68d2"
server_key_version: "encryptionkey@0"
I20260812 01:19:19.546682 930 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 01:19:19.547500 950 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:19.547672 930 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 01:19:19.547780 930 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786497534195564-24578-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786497534195564-24578-0/minicluster-data/master-0/wal
uuid: "4669bcb72fb848c0a3c1d0fbc85976ea"
format_stamp: "Formatted at 2026-08-12 01:19:19 on dist-test-slave-t3q3"
server_key: "ce0182431e7f00a7f4cf661ce570f3fd"
server_key_iv: "bc2606f3b0a4a50366823aa4cffc68d2"
server_key_version: "encryptionkey@0"
I20260812 01:19:19.547856 930 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786497534195564-24578-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786497534195564-24578-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786497534195564-24578-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:19:19.557699 930 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:19:19.557993 930 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:19:19.558159 930 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:19:19.562356 930 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.190:45555
I20260812 01:19:19.562388 1038 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.190:45555 every 8 connection(s)
I20260812 01:19:19.562760 930 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
I20260812 01:19:19.563791 1040 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:19:19.566187 1040 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4669bcb72fb848c0a3c1d0fbc85976ea: Bootstrap starting.
I20260812 01:19:19.566829 1040 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4669bcb72fb848c0a3c1d0fbc85976ea: Neither blocks nor log segments found. Creating new log.
I20260812 01:19:19.567139 1040 log.cc:824] T 00000000000000000000000000000000 P 4669bcb72fb848c0a3c1d0fbc85976ea: Log is configured to *not* fsync() on all Append() calls
I20260812 01:19:19.567893 1040 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4669bcb72fb848c0a3c1d0fbc85976ea: No bootstrap required, opened a new log
I20260812 01:19:19.569578 1040 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4669bcb72fb848c0a3c1d0fbc85976ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4669bcb72fb848c0a3c1d0fbc85976ea" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 45555 } }
I20260812 01:19:19.569726 1040 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4669bcb72fb848c0a3c1d0fbc85976ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:19:19.569749 1040 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4669bcb72fb848c0a3c1d0fbc85976ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4669bcb72fb848c0a3c1d0fbc85976ea, State: Initialized, Role: FOLLOWER
I20260812 01:19:19.569897 1040 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4669bcb72fb848c0a3c1d0fbc85976ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4669bcb72fb848c0a3c1d0fbc85976ea" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 45555 } }
I20260812 01:19:19.569981 1040 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4669bcb72fb848c0a3c1d0fbc85976ea [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:19:19.570024 1040 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4669bcb72fb848c0a3c1d0fbc85976ea [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:19:19.570076 1040 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4669bcb72fb848c0a3c1d0fbc85976ea [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:19:19.570597 1040 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4669bcb72fb848c0a3c1d0fbc85976ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4669bcb72fb848c0a3c1d0fbc85976ea" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 45555 } }
I20260812 01:19:19.570720 1040 leader_election.cc:302] T 00000000000000000000000000000000 P 4669bcb72fb848c0a3c1d0fbc85976ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4669bcb72fb848c0a3c1d0fbc85976ea; no voters:
I20260812 01:19:19.570910 1040 leader_election.cc:288] T 00000000000000000000000000000000 P 4669bcb72fb848c0a3c1d0fbc85976ea [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 01:19:19.571082 1044 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4669bcb72fb848c0a3c1d0fbc85976ea [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:19:19.571249 1040 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4669bcb72fb848c0a3c1d0fbc85976ea [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:19:19.571249 1044 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4669bcb72fb848c0a3c1d0fbc85976ea [term 1 LEADER]: Becoming Leader. State: Replica: 4669bcb72fb848c0a3c1d0fbc85976ea, State: Running, Role: LEADER
I20260812 01:19:19.571393 1044 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4669bcb72fb848c0a3c1d0fbc85976ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4669bcb72fb848c0a3c1d0fbc85976ea" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 45555 } }
I20260812 01:19:19.571847 1048 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4669bcb72fb848c0a3c1d0fbc85976ea [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4669bcb72fb848c0a3c1d0fbc85976ea. Latest consensus state: current_term: 1 leader_uuid: "4669bcb72fb848c0a3c1d0fbc85976ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4669bcb72fb848c0a3c1d0fbc85976ea" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 45555 } } }
I20260812 01:19:19.571936 1048 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4669bcb72fb848c0a3c1d0fbc85976ea [sys.catalog]: This master's current role is: LEADER
I20260812 01:19:19.572010 1046 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4669bcb72fb848c0a3c1d0fbc85976ea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4669bcb72fb848c0a3c1d0fbc85976ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4669bcb72fb848c0a3c1d0fbc85976ea" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 45555 } } }
I20260812 01:19:19.572141 1046 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4669bcb72fb848c0a3c1d0fbc85976ea [sys.catalog]: This master's current role is: LEADER
I20260812 01:19:19.572198 1062 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:19:19.572762 1062 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:19:19.572978 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 930
I20260812 01:19:19.573098 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786497534195564-24578-0/minicluster-data/master-0/wal/instance
I20260812 01:19:19.573328 24578 external_mini_cluster.cc:1428] Setting key e42ba86934552a8ddee54c36cf5ad9d7
I20260812 01:19:19.574409 1062 catalog_manager.cc:1347] Generated new cluster ID: 36fbda9a0cab4b6497d1591ff9305551
I20260812 01:19:19.574465 1062 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 01:19:19.579874 1083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:19.573924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42146 (local address 127.24.0.190:45555)
0812 01:19:19.574527 (+ 603us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:19.574536 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:19.574571 (+ 35us) server_negotiation.cc:407] Connection header received
0812 01:19:19.574699 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:19.574706 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:19.574774 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:19.574923 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:19.575174 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:19.576535 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:19.577624 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:19.578271 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:19.578409 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:19.579130 (+ 721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:19.579135 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:19.579135 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:19:19.579351 (+ 216us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:19.579396 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:19:19.579413 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:19.579489 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:19.579569 (+ 80us) server_negotiation.cc:299] Negotiation successful
0812 01:19:19.579648 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":488,"thread_start_us":83,"threads_started":1}
I20260812 01:19:19.580901 24578 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 01:19:19.586365 1062 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:19:19.586881 1062 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:19:19.593731 1062 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4669bcb72fb848c0a3c1d0fbc85976ea: Generated new TSK 0
I20260812 01:19:19.593945 1062 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 01:19:19.685535 1083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:19.601744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42150 (local address 127.24.0.190:45555)
0812 01:19:19.601793 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:19.601795 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:19.601984 (+ 189us) server_negotiation.cc:407] Connection header received
0812 01:19:19.602029 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:19.602030 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:19.602041 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:19.602109 (+ 68us) server_negotiation.cc:226] Negotiated authn=JWT
0812 01:19:19.602271 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:19.602795 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:19.603963 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:19.604246 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:19.604309 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:19.604504 (+ 195us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 01:19:19.685309 (+ 80805us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0812 01:19:19.685476 (+ 167us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:42150: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":21}
W20260812 01:19:19.685637 1083 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:42150: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260812 01:19:19.685760 1093 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:19:19.601683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:45555 (local address 127.0.0.1:42150)
0812 01:19:19.601833 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:19.601843 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:19.601906 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:19.602134 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:19.602135 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:19.602137 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0812 01:19:19.602199 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:19.602202 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:19.602878 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:19.602879 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:19.603861 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:19.603864 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:19.604330 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:19.604331 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:19.604396 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:19.604402 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 01:19:19.685474 (+ 81072us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0812 01:19:19.685627 (+ 153us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.24.0.190:45555: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":90,"threads_started":1}
I20260812 01:19:19.686136 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 930
2026-08-12T01:19:19Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (275 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (93 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (92 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 01:19:19 dist-test-slave-t3q3 krb5kdc[1114](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 01:19:19 dist-test-slave-t3q3 krb5kdc[1114](info): set up 2 sockets
Aug 12 01:19:19 dist-test-slave-t3q3 krb5kdc[1114](info): commencing operation
krb5kdc: starting...
W20260812 01:19:21.172827 24578 mini_kdc.cc:121] Time spent starting KDC: real 1.272s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
W20260812 01:19:21.329084 24578 mini_kdc.cc:253] Time spent creating user principal test-admin: real 0.156s user 0.000s sys 0.002s
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
W20260812 01:19:21.487663 24578 mini_kdc.cc:253] Time spent creating user principal test-user: real 0.158s user 0.000s sys 0.002s
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
W20260812 01:19:21.609150 24578 mini_kdc.cc:253] Time spent creating user principal joe-interloper: real 0.121s user 0.000s sys 0.002s
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/krb5kdc/test-user.keytab.
W20260812 01:19:22.161253 24578 mini_kdc.cc:302] Time spent creating keytab for test-user: real 0.552s user 0.000s sys 0.002s
Aug 12 01:19:22 dist-test-slave-t3q3 krb5kdc[1114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497562, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T01:19:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:19:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.0.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.0.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.0.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab.
Entry for principal kudu/127.24.0.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab.
W20260812 01:19:22.917114 24578 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.24.0.190: real 0.731s user 0.001s sys 0.003s
I20260812 01:19:22.917550 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.190:37961
--webserver_interface=127.24.0.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab
--principal=kudu/127.24.0.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:35899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.0.190:37961
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:19:23.000806 1133 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:19:23.001003 1133 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:19:23.001025 1133 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:19:23.002267 1133 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:19:23.002314 1133 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:19:23.002328 1133 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:19:23.002341 1133 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:19:23.002354 1133 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:19:23.003846 1133 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:35899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.0.190:37961
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab
--principal=kudu/127.24.0.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.190:37961
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.0.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1133
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:19:23.004122 1133 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:19:23.004400 1133 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:19:23.009604 1143 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:23.009627 1144 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:23.009604 1147 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:19:23.009804 1133 server_base.cc:1034] running on GCE node
I20260812 01:19:23.010125 1133 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:19:23.010408 1133 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:19:23.011554 1133 hybrid_clock.cc:648] HybridClock initialized: now 1786497563011534 us; error 37 us; skew 500 ppm
Aug 12 01:19:23 dist-test-slave-t3q3 krb5kdc[1114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497563, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:19:23.014225 1133 init.cc:377] Logged in from keytab as kudu/127.24.0.190@KRBTEST.COM (short username kudu)
I20260812 01:19:23.014605 1133 webserver.cc:460] Webserver started at http://127.24.0.190:40163/ using document root <none> and password file <none>
I20260812 01:19:23.014807 1133 fs_manager.cc:359] Metadata directory not provided
I20260812 01:19:23.014848 1133 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:19:23.014937 1133 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:19:23.015933 1133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/minicluster-data/master-0/data/instance:
uuid: "397a6babe49a48d89f418e96dd4b99fe"
format_stamp: "Formatted at 2026-08-12 01:19:23 on dist-test-slave-t3q3"
server_key: "5ba7665858f27706119b21916d74dbeb"
server_key_iv: "dd9630612ad0d1e242d711e782ea049a"
server_key_version: "encryptionkey@0"
I20260812 01:19:23.016311 1133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/minicluster-data/master-0/wal/instance:
uuid: "397a6babe49a48d89f418e96dd4b99fe"
format_stamp: "Formatted at 2026-08-12 01:19:23 on dist-test-slave-t3q3"
server_key: "5ba7665858f27706119b21916d74dbeb"
server_key_iv: "dd9630612ad0d1e242d711e782ea049a"
server_key_version: "encryptionkey@0"
I20260812 01:19:23.017550 1133 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 01:19:23.018226 1161 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:23.018393 1133 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 01:19:23.018483 1133 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/minicluster-data/master-0/wal
uuid: "397a6babe49a48d89f418e96dd4b99fe"
format_stamp: "Formatted at 2026-08-12 01:19:23 on dist-test-slave-t3q3"
server_key: "5ba7665858f27706119b21916d74dbeb"
server_key_iv: "dd9630612ad0d1e242d711e782ea049a"
server_key_version: "encryptionkey@0"
I20260812 01:19:23.018551 1133 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:19:23.033751 1133 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:19:23.036250 1133 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:19:23.036428 1133 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:19:23.040432 1133 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.190:37961
I20260812 01:19:23.040504 1246 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.190:37961 every 8 connection(s)
I20260812 01:19:23.040894 1133 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
I20260812 01:19:23.041808 1247 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:19:23.043509 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 1133
I20260812 01:19:23.043632 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/minicluster-data/master-0/wal/instance
I20260812 01:19:23.043846 24578 external_mini_cluster.cc:1428] Setting key 718d4c7272d85d2c3bb10bbb475ef1c1
I20260812 01:19:23.044621 1247 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 397a6babe49a48d89f418e96dd4b99fe: Bootstrap starting.
I20260812 01:19:23.045238 1247 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 397a6babe49a48d89f418e96dd4b99fe: Neither blocks nor log segments found. Creating new log.
I20260812 01:19:23.045567 1247 log.cc:824] T 00000000000000000000000000000000 P 397a6babe49a48d89f418e96dd4b99fe: Log is configured to *not* fsync() on all Append() calls
I20260812 01:19:23.046398 1247 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 397a6babe49a48d89f418e96dd4b99fe: No bootstrap required, opened a new log
I20260812 01:19:23.048267 1247 raft_consensus.cc:348] T 00000000000000000000000000000000 P 397a6babe49a48d89f418e96dd4b99fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "397a6babe49a48d89f418e96dd4b99fe" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 37961 } }
I20260812 01:19:23.048424 1247 raft_consensus.cc:374] T 00000000000000000000000000000000 P 397a6babe49a48d89f418e96dd4b99fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:19:23.048511 1247 raft_consensus.cc:729] T 00000000000000000000000000000000 P 397a6babe49a48d89f418e96dd4b99fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 397a6babe49a48d89f418e96dd4b99fe, State: Initialized, Role: FOLLOWER
I20260812 01:19:23.048681 1247 consensus_queue.cc:260] T 00000000000000000000000000000000 P 397a6babe49a48d89f418e96dd4b99fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "397a6babe49a48d89f418e96dd4b99fe" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 37961 } }
I20260812 01:19:23.048779 1247 raft_consensus.cc:388] T 00000000000000000000000000000000 P 397a6babe49a48d89f418e96dd4b99fe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:19:23.048822 1247 raft_consensus.cc:482] T 00000000000000000000000000000000 P 397a6babe49a48d89f418e96dd4b99fe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:19:23.048871 1247 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 397a6babe49a48d89f418e96dd4b99fe [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:19:23.049396 1247 raft_consensus.cc:504] T 00000000000000000000000000000000 P 397a6babe49a48d89f418e96dd4b99fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "397a6babe49a48d89f418e96dd4b99fe" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 37961 } }
I20260812 01:19:23.049515 1247 leader_election.cc:302] T 00000000000000000000000000000000 P 397a6babe49a48d89f418e96dd4b99fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 397a6babe49a48d89f418e96dd4b99fe; no voters:
I20260812 01:19:23.049736 1247 leader_election.cc:288] T 00000000000000000000000000000000 P 397a6babe49a48d89f418e96dd4b99fe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 01:19:23.049772 1255 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 397a6babe49a48d89f418e96dd4b99fe [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:19:23.049897 1255 raft_consensus.cc:686] T 00000000000000000000000000000000 P 397a6babe49a48d89f418e96dd4b99fe [term 1 LEADER]: Becoming Leader. State: Replica: 397a6babe49a48d89f418e96dd4b99fe, State: Running, Role: LEADER
I20260812 01:19:23.049983 1255 consensus_queue.cc:237] T 00000000000000000000000000000000 P 397a6babe49a48d89f418e96dd4b99fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "397a6babe49a48d89f418e96dd4b99fe" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 37961 } }
I20260812 01:19:23.050016 1247 sys_catalog.cc:564] T 00000000000000000000000000000000 P 397a6babe49a48d89f418e96dd4b99fe [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:19:23.050374 1257 sys_catalog.cc:455] T 00000000000000000000000000000000 P 397a6babe49a48d89f418e96dd4b99fe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "397a6babe49a48d89f418e96dd4b99fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "397a6babe49a48d89f418e96dd4b99fe" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 37961 } } }
I20260812 01:19:23.050454 1257 sys_catalog.cc:458] T 00000000000000000000000000000000 P 397a6babe49a48d89f418e96dd4b99fe [sys.catalog]: This master's current role is: LEADER
Aug 12 01:19:23 dist-test-slave-t3q3 krb5kdc[1114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497562, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
I20260812 01:19:23.050668 1261 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:19:23.051211 1261 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:19:23.051646 1256 sys_catalog.cc:455] T 00000000000000000000000000000000 P 397a6babe49a48d89f418e96dd4b99fe [sys.catalog]: SysCatalogTable state changed. Reason: New leader 397a6babe49a48d89f418e96dd4b99fe. Latest consensus state: current_term: 1 leader_uuid: "397a6babe49a48d89f418e96dd4b99fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "397a6babe49a48d89f418e96dd4b99fe" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 37961 } } }
I20260812 01:19:23.051723 1256 sys_catalog.cc:458] T 00000000000000000000000000000000 P 397a6babe49a48d89f418e96dd4b99fe [sys.catalog]: This master's current role is: LEADER
I20260812 01:19:23.053864 1280 mvcc.cc:205] Tried to move back new op lower bound from 7317494018264264704 to 7317494018253279232. Current Snapshot: MvccSnapshot[applied={T|T < 7317494018264264704}]
I20260812 01:19:23.054344 1261 catalog_manager.cc:1347] Generated new cluster ID: bc840b0c7ab8490eb99f0adc28456624
I20260812 01:19:23.054394 1261 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 01:19:23.055272 1251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:23.044451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35442 (local address 127.24.0.190:37961)
0812 01:19:23.044639 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:23.044649 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:23.044689 (+ 40us) server_negotiation.cc:407] Connection header received
0812 01:19:23.045160 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:23.045169 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:23.045216 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:23.045375 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:23.046122 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:23.047071 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:23.047839 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:23.048307 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:23.048425 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:23.051296 (+ 2871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:23.051301 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:23.051301 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:23.051313 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:23.053330 (+ 2017us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:23.053802 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:23.053806 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:23.053807 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:23.053914 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:23.054147 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:23.054148 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:23.054149 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:23.054467 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:23.054616 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:23.054882 (+ 266us) server_negotiation.cc:299] Negotiation successful
0812 01:19:23.055031 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":93,"threads_started":1}
I20260812 01:19:23.071821 1261 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:19:23.072511 1261 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:19:23.084158 1261 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 397a6babe49a48d89f418e96dd4b99fe: Generated new TSK 0
I20260812 01:19:23.084407 1261 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.0.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.0.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.0.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.129.keytab.
Entry for principal kudu/127.24.0.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.129.keytab.
I20260812 01:19:23.119702 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.0.129:0
--local_ip_for_outbound_sockets=127.24.0.129
--webserver_interface=127.24.0.129
--webserver_port=0
--tserver_master_addrs=127.24.0.190:37961
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.129.keytab
--principal=kudu/127.24.0.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:35899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:19:23.226346 1288 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:19:23.226586 1288 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:19:23.226611 1288 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:19:23.227730 1288 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:19:23.227771 1288 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:19:23.227820 1288 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.0.129
I20260812 01:19:23.229349 1288 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:35899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.129.keytab
--principal=kudu/127.24.0.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.0.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.0.129
--webserver_port=0
--tserver_master_addrs=127.24.0.190:37961
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1288
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.0.129
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:19:23.229584 1288 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:19:23.229825 1288 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:19:23.234807 1301 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:23.234905 1298 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:19:23.235054 1288 server_base.cc:1034] running on GCE node
W20260812 01:19:23.234805 1297 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:19:23.235342 1288 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:19:23.235546 1288 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:19:23.236681 1288 hybrid_clock.cc:648] HybridClock initialized: now 1786497563236671 us; error 30 us; skew 500 ppm
Aug 12 01:19:23 dist-test-slave-t3q3 krb5kdc[1114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497563, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:19:23.239135 1288 init.cc:377] Logged in from keytab as kudu/127.24.0.129@KRBTEST.COM (short username kudu)
I20260812 01:19:23.239526 1288 webserver.cc:460] Webserver started at http://127.24.0.129:33585/ using document root <none> and password file <none>
I20260812 01:19:23.239698 1288 fs_manager.cc:359] Metadata directory not provided
I20260812 01:19:23.239738 1288 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:19:23.239826 1288 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:19:23.240841 1288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/minicluster-data/ts-0/data/instance:
uuid: "727d9671f24640388c695297b99a5151"
format_stamp: "Formatted at 2026-08-12 01:19:23 on dist-test-slave-t3q3"
server_key: "34cb024ed58e408ca8d1bed15bef2695"
server_key_iv: "0b6f23101a965af631c9cf4ec8549c3d"
server_key_version: "encryptionkey@0"
I20260812 01:19:23.241137 1288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/minicluster-data/ts-0/wal/instance:
uuid: "727d9671f24640388c695297b99a5151"
format_stamp: "Formatted at 2026-08-12 01:19:23 on dist-test-slave-t3q3"
server_key: "34cb024ed58e408ca8d1bed15bef2695"
server_key_iv: "0b6f23101a965af631c9cf4ec8549c3d"
server_key_version: "encryptionkey@0"
I20260812 01:19:23.242408 1288 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 01:19:23.243085 1310 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:23.243270 1288 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 01:19:23.243410 1288 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/minicluster-data/ts-0/wal
uuid: "727d9671f24640388c695297b99a5151"
format_stamp: "Formatted at 2026-08-12 01:19:23 on dist-test-slave-t3q3"
server_key: "34cb024ed58e408ca8d1bed15bef2695"
server_key_iv: "0b6f23101a965af631c9cf4ec8549c3d"
server_key_version: "encryptionkey@0"
I20260812 01:19:23.243511 1288 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:19:23.274632 1288 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:19:23.276798 1288 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:19:23.276973 1288 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:19:23.277249 1288 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:19:23.277590 1288 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:19:23.277647 1288 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:23.277683 1288 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:19:23.277714 1288 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:23.283612 1288 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.129:38479
I20260812 01:19:23.283664 1497 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.129:38479 every 8 connection(s)
I20260812 01:19:23.284018 1288 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/minicluster-data/ts-0/data/info.pb
I20260812 01:19:23.286922 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 1288
I20260812 01:19:23.287065 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/minicluster-data/ts-0/wal/instance
I20260812 01:19:23.287268 24578 external_mini_cluster.cc:1428] Setting key 1ee12864ffa46aa682fb94fb71c50cbf
Aug 12 01:19:23 dist-test-slave-t3q3 krb5kdc[1114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497563, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.129@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
I20260812 01:19:23.294651 1251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:23.284866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.129:34339 (local address 127.24.0.190:37961)
0812 01:19:23.284949 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:23.284951 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:23.285124 (+ 173us) server_negotiation.cc:407] Connection header received
0812 01:19:23.286046 (+ 922us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:23.286047 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:23.286059 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:23.286134 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:23.286712 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:23.287385 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:23.288664 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:23.288895 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:23.288973 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:23.291609 (+ 2636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:23.291610 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:23.291610 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:23.291617 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:23.293063 (+ 1446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:23.293558 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:23.293559 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:23.293559 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:23.293621 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:23.293947 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:23.293948 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:23.293948 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:23.294078 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:23.294144 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:23.294510 (+ 366us) server_negotiation.cc:299] Negotiation successful
0812 01:19:23.294587 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 01:19:23.294961 1500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:23.284809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:37961 (local address 127.24.0.129:34339)
0812 01:19:23.284993 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:23.285031 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:23.285949 (+ 918us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:23.286196 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:23.286199 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:23.286440 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:23.286628 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:23.286633 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:23.287500 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:23.287502 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:23.288550 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:23.288554 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:23.288994 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:23.288996 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:23.289129 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:23.289784 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:23.289795 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:23.291512 (+ 1717us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:19:23.293143 (+ 1631us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:23.293152 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:23.293154 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:23.293481 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:23.293691 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:23.293692 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:23.293693 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:23.293843 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:23.294169 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:19:23.294176 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:19:23.294355 (+ 179us) client_negotiation.cc:769] Sending connection context
0812 01:19:23.294489 (+ 134us) client_negotiation.cc:240] Negotiation successful
0812 01:19:23.294676 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":97,"threads_started":1}
I20260812 01:19:23.295621 1498 heartbeater.cc:347] Connected to a master server at 127.24.0.190:37961
I20260812 01:19:23.295729 1498 heartbeater.cc:464] Registering TS with master...
I20260812 01:19:23.295931 1498 heartbeater.cc:511] Master 127.24.0.190:37961 requested a full tablet report, sending...
I20260812 01:19:23.296581 1185 ts_manager.cc:194] Registered new tserver with Master: 727d9671f24640388c695297b99a5151 (127.24.0.129:38479)
I20260812 01:19:23.297561 1185 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.0.129@KRBTEST.COM'} at 127.24.0.129:34339
WARNING: no policy specified for kudu/127.24.0.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.0.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.0.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.130.keytab.
Entry for principal kudu/127.24.0.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.130.keytab.
I20260812 01:19:23.316906 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.0.130:0
--local_ip_for_outbound_sockets=127.24.0.130
--webserver_interface=127.24.0.130
--webserver_port=0
--tserver_master_addrs=127.24.0.190:37961
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.130.keytab
--principal=kudu/127.24.0.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:35899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:19:23.400528 1504 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:19:23.400777 1504 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:19:23.400815 1504 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:19:23.401958 1504 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:19:23.402040 1504 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:19:23.402086 1504 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.0.130
I20260812 01:19:23.403561 1504 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:35899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.130.keytab
--principal=kudu/127.24.0.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.0.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.0.130
--webserver_port=0
--tserver_master_addrs=127.24.0.190:37961
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1504
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.0.130
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:19:23.403855 1504 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:19:23.404126 1504 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:19:23.409394 1516 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:23.409409 1510 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:23.409394 1512 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:19:23.409485 1504 server_base.cc:1034] running on GCE node
I20260812 01:19:23.409863 1504 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:19:23.410132 1504 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:19:23.411266 1504 hybrid_clock.cc:648] HybridClock initialized: now 1786497563411252 us; error 30 us; skew 500 ppm
Aug 12 01:19:23 dist-test-slave-t3q3 krb5kdc[1114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497563, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:19:23.413849 1504 init.cc:377] Logged in from keytab as kudu/127.24.0.130@KRBTEST.COM (short username kudu)
I20260812 01:19:23.414342 1504 webserver.cc:460] Webserver started at http://127.24.0.130:43759/ using document root <none> and password file <none>
I20260812 01:19:23.414578 1504 fs_manager.cc:359] Metadata directory not provided
I20260812 01:19:23.414657 1504 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:19:23.414819 1504 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:19:23.415920 1504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/minicluster-data/ts-1/data/instance:
uuid: "92d2281e01574b95bd21cf6d5bdef517"
format_stamp: "Formatted at 2026-08-12 01:19:23 on dist-test-slave-t3q3"
server_key: "cfbef75bcfe4aff105101f518a54fe4e"
server_key_iv: "256df4ab9a674bddf3211b0cfe28a48a"
server_key_version: "encryptionkey@0"
I20260812 01:19:23.416610 1504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/minicluster-data/ts-1/wal/instance:
uuid: "92d2281e01574b95bd21cf6d5bdef517"
format_stamp: "Formatted at 2026-08-12 01:19:23 on dist-test-slave-t3q3"
server_key: "cfbef75bcfe4aff105101f518a54fe4e"
server_key_iv: "256df4ab9a674bddf3211b0cfe28a48a"
server_key_version: "encryptionkey@0"
I20260812 01:19:23.418072 1504 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 01:19:23.418860 1531 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:23.419124 1504 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:19:23.419234 1504 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/minicluster-data/ts-1/wal
uuid: "92d2281e01574b95bd21cf6d5bdef517"
format_stamp: "Formatted at 2026-08-12 01:19:23 on dist-test-slave-t3q3"
server_key: "cfbef75bcfe4aff105101f518a54fe4e"
server_key_iv: "256df4ab9a674bddf3211b0cfe28a48a"
server_key_version: "encryptionkey@0"
I20260812 01:19:23.419312 1504 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:19:23.433739 1504 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:19:23.435745 1504 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:19:23.435918 1504 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:19:23.436233 1504 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:19:23.436582 1504 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:19:23.436640 1504 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:23.436683 1504 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:19:23.436710 1504 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:23.442903 1504 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.130:41067
I20260812 01:19:23.443004 1716 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.130:41067 every 8 connection(s)
I20260812 01:19:23.443326 1504 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/minicluster-data/ts-1/data/info.pb
Aug 12 01:19:23 dist-test-slave-t3q3 krb5kdc[1114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497563, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.130@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
I20260812 01:19:23.452188 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 1504
I20260812 01:19:23.452333 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/minicluster-data/ts-1/wal/instance
I20260812 01:19:23.452549 24578 external_mini_cluster.cc:1428] Setting key e594dd71e5ce85db2f3a357ba07ed464
I20260812 01:19:23.453529 1251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:23.444136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.130:36405 (local address 127.24.0.190:37961)
0812 01:19:23.444212 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:23.444215 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:23.444431 (+ 216us) server_negotiation.cc:407] Connection header received
0812 01:19:23.445418 (+ 987us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:23.445420 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:23.445431 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:23.445496 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:23.446129 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:23.446745 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:23.447828 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:23.448011 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:23.448123 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:23.450625 (+ 2502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:23.450626 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:23.450627 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:23.450634 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:23.451996 (+ 1362us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:23.452489 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:23.452491 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:23.452491 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:23.452560 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:23.452845 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:23.452846 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:23.452846 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:23.452971 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:23.453062 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:23.453351 (+ 289us) server_negotiation.cc:299] Negotiation successful
0812 01:19:23.453437 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 01:19:23.453753 1721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:23.444101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:37961 (local address 127.24.0.130:36405)
0812 01:19:23.444302 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:23.444340 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:23.445289 (+ 949us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:23.445574 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:23.445577 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:23.445844 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:23.446046 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:23.446050 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:23.446829 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:23.446831 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:23.447754 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:23.447757 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:23.448125 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:23.448126 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:23.448240 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:23.448809 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:23.448818 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:23.450490 (+ 1672us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:19:23.452110 (+ 1620us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:23.452118 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:23.452121 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:23.452413 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:23.452641 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:23.452641 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:23.452642 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:23.452764 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:23.453053 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:19:23.453060 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:19:23.453209 (+ 149us) client_negotiation.cc:769] Sending connection context
0812 01:19:23.453317 (+ 108us) client_negotiation.cc:240] Negotiation successful
0812 01:19:23.453489 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":112,"threads_started":1}
I20260812 01:19:23.454345 1719 heartbeater.cc:347] Connected to a master server at 127.24.0.190:37961
I20260812 01:19:23.454476 1719 heartbeater.cc:464] Registering TS with master...
I20260812 01:19:23.454666 1719 heartbeater.cc:511] Master 127.24.0.190:37961 requested a full tablet report, sending...
I20260812 01:19:23.455116 1185 ts_manager.cc:194] Registered new tserver with Master: 92d2281e01574b95bd21cf6d5bdef517 (127.24.0.130:41067)
I20260812 01:19:23.455567 1185 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.0.130@KRBTEST.COM'} at 127.24.0.130:36405
WARNING: no policy specified for kudu/127.24.0.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.0.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.0.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.131.keytab.
Entry for principal kudu/127.24.0.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.131.keytab.
I20260812 01:19:23.478863 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.0.131:0
--local_ip_for_outbound_sockets=127.24.0.131
--webserver_interface=127.24.0.131
--webserver_port=0
--tserver_master_addrs=127.24.0.190:37961
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.131.keytab
--principal=kudu/127.24.0.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:35899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:19:23.563060 1725 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:19:23.563311 1725 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:19:23.563354 1725 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:19:23.564677 1725 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:19:23.564723 1725 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:19:23.564749 1725 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.0.131
I20260812 01:19:23.566267 1725 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:35899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.131.keytab
--principal=kudu/127.24.0.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.0.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.0.131
--webserver_port=0
--tserver_master_addrs=127.24.0.190:37961
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1725
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.0.131
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:19:23.566520 1725 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:19:23.566751 1725 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:19:23.571730 1733 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:23.571743 1734 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:19:23.571889 1725 server_base.cc:1034] running on GCE node
W20260812 01:19:23.571743 1736 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:19:23.572273 1725 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:19:23.572569 1725 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:19:23.573755 1725 hybrid_clock.cc:648] HybridClock initialized: now 1786497563573707 us; error 52 us; skew 500 ppm
Aug 12 01:19:23 dist-test-slave-t3q3 krb5kdc[1114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497563, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:19:23.576295 1725 init.cc:377] Logged in from keytab as kudu/127.24.0.131@KRBTEST.COM (short username kudu)
I20260812 01:19:23.576756 1725 webserver.cc:460] Webserver started at http://127.24.0.131:46625/ using document root <none> and password file <none>
I20260812 01:19:23.576948 1725 fs_manager.cc:359] Metadata directory not provided
I20260812 01:19:23.576992 1725 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:19:23.577081 1725 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:19:23.578063 1725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/minicluster-data/ts-2/data/instance:
uuid: "73e46956016a4144a0a24985616b8ff6"
format_stamp: "Formatted at 2026-08-12 01:19:23 on dist-test-slave-t3q3"
server_key: "82fd85ae9a5635541ffd0f552fc0e4f1"
server_key_iv: "301c8ca21c9448d037ea5f9d62335d51"
server_key_version: "encryptionkey@0"
I20260812 01:19:23.578382 1725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/minicluster-data/ts-2/wal/instance:
uuid: "73e46956016a4144a0a24985616b8ff6"
format_stamp: "Formatted at 2026-08-12 01:19:23 on dist-test-slave-t3q3"
server_key: "82fd85ae9a5635541ffd0f552fc0e4f1"
server_key_iv: "301c8ca21c9448d037ea5f9d62335d51"
server_key_version: "encryptionkey@0"
I20260812 01:19:23.579629 1725 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 01:19:23.580368 1753 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:23.580545 1725 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:19:23.580652 1725 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/minicluster-data/ts-2/wal
uuid: "73e46956016a4144a0a24985616b8ff6"
format_stamp: "Formatted at 2026-08-12 01:19:23 on dist-test-slave-t3q3"
server_key: "82fd85ae9a5635541ffd0f552fc0e4f1"
server_key_iv: "301c8ca21c9448d037ea5f9d62335d51"
server_key_version: "encryptionkey@0"
I20260812 01:19:23.580731 1725 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:19:23.596347 1725 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:19:23.598255 1725 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:19:23.598421 1725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:19:23.598650 1725 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:19:23.598974 1725 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:19:23.599030 1725 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:23.599071 1725 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:19:23.599097 1725 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:23.605245 1725 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.131:38475
I20260812 01:19:23.605335 1926 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.131:38475 every 8 connection(s)
I20260812 01:19:23.605674 1725 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/minicluster-data/ts-2/data/info.pb
Aug 12 01:19:23 dist-test-slave-t3q3 krb5kdc[1114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497563, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.131@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
I20260812 01:19:23.614434 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 1725
I20260812 01:19:23.614537 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/minicluster-data/ts-2/wal/instance
I20260812 01:19:23.614748 24578 external_mini_cluster.cc:1428] Setting key a8d7af84b07c1f7e35d7257f05eacedb
I20260812 01:19:23.615260 1251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:23.606449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.131:34411 (local address 127.24.0.190:37961)
0812 01:19:23.606509 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:23.606511 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:23.606725 (+ 214us) server_negotiation.cc:407] Connection header received
0812 01:19:23.607730 (+ 1005us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:23.607731 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:23.607745 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:23.607796 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:23.608531 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:23.609040 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:23.610027 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:23.610226 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:23.610303 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:23.612376 (+ 2073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:23.612377 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:23.612378 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:23.612384 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:23.613819 (+ 1435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:23.614165 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:23.614165 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:23.614166 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:23.614217 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:23.614543 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:23.614544 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:23.614545 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:23.614684 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:23.614739 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:23.615081 (+ 342us) server_negotiation.cc:299] Negotiation successful
0812 01:19:23.615172 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 01:19:23.615449 1929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:23.606425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:37961 (local address 127.24.0.131:34411)
0812 01:19:23.606600 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:23.606635 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:23.607597 (+ 962us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:23.607873 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:23.607876 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:23.608229 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:23.608430 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:23.608435 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:23.609140 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:23.609141 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:23.609932 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:23.609934 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:23.610325 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:23.610326 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:23.610442 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:23.610890 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:23.610899 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:23.612289 (+ 1390us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:19:23.613890 (+ 1601us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:23.613895 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:23.613897 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:23.614099 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:23.614300 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:23.614300 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:23.614301 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:23.614443 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:23.614754 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:19:23.614761 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:19:23.614930 (+ 169us) client_negotiation.cc:769] Sending connection context
0812 01:19:23.615049 (+ 119us) client_negotiation.cc:240] Negotiation successful
0812 01:19:23.615209 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":95,"threads_started":1}
I20260812 01:19:23.615960 1927 heartbeater.cc:347] Connected to a master server at 127.24.0.190:37961
I20260812 01:19:23.616091 1927 heartbeater.cc:464] Registering TS with master...
I20260812 01:19:23.616269 1927 heartbeater.cc:511] Master 127.24.0.190:37961 requested a full tablet report, sending...
I20260812 01:19:23.616648 1185 ts_manager.cc:194] Registered new tserver with Master: 73e46956016a4144a0a24985616b8ff6 (127.24.0.131:38475)
I20260812 01:19:23.617018 1185 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.0.131@KRBTEST.COM'} at 127.24.0.131:34411
I20260812 01:19:23.626457 24578 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/krb5kdc/test-admin.keytab.
Aug 12 01:19:23 dist-test-slave-t3q3 krb5kdc[1114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497563, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:19:23.643865 24578 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260812 01:19:23.645843 1940 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:19:23.644894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41700 (local address 127.24.0.129:38479)
0812 01:19:23.645131 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:23.645135 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:23.645164 (+ 29us) server_negotiation.cc:407] Connection header received
0812 01:19:23.645193 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:23.645195 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:23.645214 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:23.645301 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:23.645736 (+ 435us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41700: BlockingRecv error: recv got EOF from 127.0.0.1:41700 (error 108)
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":123,"threads_started":1}
W20260812 01:19:23.646000 24578 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/krb5kdc/krb5cc' of type FILE
Aug 12 01:19:23 dist-test-slave-t3q3 krb5kdc[1114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497563, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:19:23.647092 24578 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 12 01:19:23 dist-test-slave-t3q3 krb5kdc[1114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497563, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:19:23.648038 24578 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 12 01:19:23 dist-test-slave-t3q3 krb5kdc[1114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497563, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.0.129@KRBTEST.COM
I20260812 01:19:23.656512 1940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:23.648579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41706 (local address 127.24.0.129:38479)
0812 01:19:23.648617 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:23.648619 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:23.648650 (+ 31us) server_negotiation.cc:407] Connection header received
0812 01:19:23.648712 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:23.648713 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:23.648723 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:23.648772 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:23.649141 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:23.649930 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:23.650663 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:23.651061 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:23.651135 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:23.653286 (+ 2151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:23.653296 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:23.653296 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:23.653303 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:23.655212 (+ 1909us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:23.655588 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:23.655591 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:23.655592 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:23.655656 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:23.655875 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:23.655876 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:23.655876 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:23.656021 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:23.656223 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:23.656312 (+ 89us) server_negotiation.cc:299] Negotiation successful
0812 01:19:23.656397 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
W20260812 01:19:23.658154 1940 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:19:23.657678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41710 (local address 127.24.0.129:38479)
0812 01:19:23.657723 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:23.657725 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:23.657788 (+ 63us) server_negotiation.cc:407] Connection header received
0812 01:19:23.657848 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:23.657848 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:23.657857 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:23.657901 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:23.658121 (+ 220us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41710: BlockingRecv error: recv got EOF from 127.0.0.1:41710 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260812 01:19:23.658293 24578 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/krb5kdc/krb5cc)
Aug 12 01:19:23 dist-test-slave-t3q3 krb5kdc[1114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497563, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:19:23.659225 24578 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 12 01:19:23 dist-test-slave-t3q3 krb5kdc[1114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497563, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:19:23.660156 24578 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 12 01:19:23 dist-test-slave-t3q3 krb5kdc[1114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497563, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.0.129@KRBTEST.COM
I20260812 01:19:23.666852 1940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:23.660652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41724 (local address 127.24.0.129:38479)
0812 01:19:23.660706 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:23.660708 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:23.660781 (+ 73us) server_negotiation.cc:407] Connection header received
0812 01:19:23.660826 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:23.660827 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:23.660835 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:23.660879 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:23.661222 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:23.661676 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:23.662301 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:23.662479 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:23.662527 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:23.664490 (+ 1963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:23.664492 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:23.664492 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:23.664498 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:23.665859 (+ 1361us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:23.666185 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:23.666186 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:23.666186 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:23.666233 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:23.666403 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:23.666403 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:23.666403 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:23.666484 (+ 81us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:23.666549 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:23.666643 (+ 94us) server_negotiation.cc:299] Negotiation successful
0812 01:19:23.666706 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
W20260812 01:19:23.668394 1940 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:19:23.667851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41732 (local address 127.24.0.129:38479)
0812 01:19:23.667885 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:23.667887 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:23.667925 (+ 38us) server_negotiation.cc:407] Connection header received
0812 01:19:23.667981 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:23.667981 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:23.667988 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:23.668034 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:23.668333 (+ 299us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41732: BlockingRecv error: recv got EOF from 127.0.0.1:41732 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260812 01:19:23.668483 24578 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786497534195564-24578-0/krb5kdc/krb5cc' of type FILE
Aug 12 01:19:23 dist-test-slave-t3q3 krb5kdc[1114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497563, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:19:23.669369 24578 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 12 01:19:23 dist-test-slave-t3q3 krb5kdc[1114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497563, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:19:23.670187 24578 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 12 01:19:23 dist-test-slave-t3q3 krb5kdc[1114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497563, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.0.129@KRBTEST.COM
I20260812 01:19:23.676563 1940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:23.670619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41736 (local address 127.24.0.129:38479)
0812 01:19:23.670669 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:23.670670 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:23.670681 (+ 11us) server_negotiation.cc:407] Connection header received
0812 01:19:23.670754 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:23.670755 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:23.670762 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:23.670805 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:23.671120 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:23.671530 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:23.672263 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:23.672429 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:23.672478 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:23.674303 (+ 1825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:23.674305 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:23.674305 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:23.674310 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:23.675637 (+ 1327us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:23.675915 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:23.675916 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:23.675916 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:23.675961 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:23.676168 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:23.676168 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:23.676169 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:23.676245 (+ 76us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:23.676310 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:23.676400 (+ 90us) server_negotiation.cc:299] Negotiation successful
0812 01:19:23.676450 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260812 01:19:23.677213 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 1288
I20260812 01:19:23.704118 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 1504
I20260812 01:19:23.723843 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 1725
I20260812 01:19:23.733460 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 1133
2026-08-12T01:19:23Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (3857 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 01:19:23 dist-test-slave-t3q3 krb5kdc[1963](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 01:19:23 dist-test-slave-t3q3 krb5kdc[1963](info): set up 2 sockets
Aug 12 01:19:23 dist-test-slave-t3q3 krb5kdc[1963](info): commencing operation
krb5kdc: starting...
W20260812 01:19:25.008522 24578 mini_kdc.cc:121] Time spent starting KDC: real 1.250s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/krb5kdc/test-user.keytab.
Aug 12 01:19:25 dist-test-slave-t3q3 krb5kdc[1963](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497565, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T01:19:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:19:25Z Disabled control of system clock
WARNING: no policy specified for oryx/127.24.0.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.0.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.0.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/krb5kdc/oryx_127.24.0.190.keytab.
Entry for principal oryx/127.24.0.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/krb5kdc/oryx_127.24.0.190.keytab.
I20260812 01:19:25.135622 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.190:46409
--webserver_interface=127.24.0.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/krb5kdc/oryx_127.24.0.190.keytab
--principal=oryx/127.24.0.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:38987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.0.190:46409
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:19:25.221323 1984 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:19:25.221575 1984 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:19:25.221613 1984 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:19:25.222788 1984 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:19:25.222859 1984 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:19:25.222886 1984 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:19:25.222918 1984 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:19:25.222952 1984 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:19:25.224509 1984 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:38987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.0.190:46409
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/krb5kdc/oryx_127.24.0.190.keytab
--principal=oryx/127.24.0.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.190:46409
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.0.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1984
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:19:25.224798 1984 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:19:25.225030 1984 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:19:25.229894 1997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:25.229898 1994 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:25.229910 1993 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:19:25.230186 1984 server_base.cc:1034] running on GCE node
I20260812 01:19:25.230486 1984 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:19:25.230791 1984 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:19:25.231943 1984 hybrid_clock.cc:648] HybridClock initialized: now 1786497565231920 us; error 40 us; skew 500 ppm
Aug 12 01:19:25 dist-test-slave-t3q3 krb5kdc[1963](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497565, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.0.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:19:25.237385 1984 init.cc:377] Logged in from keytab as oryx/127.24.0.190@KRBTEST.COM (short username oryx)
I20260812 01:19:25.237895 1984 webserver.cc:460] Webserver started at http://127.24.0.190:32967/ using document root <none> and password file <none>
I20260812 01:19:25.238133 1984 fs_manager.cc:359] Metadata directory not provided
I20260812 01:19:25.238204 1984 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:19:25.238364 1984 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:19:25.239449 1984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/master-0/data/instance:
uuid: "efbfb4db2c524879a5d7d17801ff4cb0"
format_stamp: "Formatted at 2026-08-12 01:19:25 on dist-test-slave-t3q3"
server_key: "59fb4b8540e0dd5ee4183a539f1cce0f"
server_key_iv: "3ae21d4bc48d3d470eeea4f054ee1ff2"
server_key_version: "encryptionkey@0"
I20260812 01:19:25.239836 1984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/master-0/wal/instance:
uuid: "efbfb4db2c524879a5d7d17801ff4cb0"
format_stamp: "Formatted at 2026-08-12 01:19:25 on dist-test-slave-t3q3"
server_key: "59fb4b8540e0dd5ee4183a539f1cce0f"
server_key_iv: "3ae21d4bc48d3d470eeea4f054ee1ff2"
server_key_version: "encryptionkey@0"
I20260812 01:19:25.241250 1984 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 01:19:25.242003 2004 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:25.242184 1984 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 01:19:25.242285 1984 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/master-0/wal
uuid: "efbfb4db2c524879a5d7d17801ff4cb0"
format_stamp: "Formatted at 2026-08-12 01:19:25 on dist-test-slave-t3q3"
server_key: "59fb4b8540e0dd5ee4183a539f1cce0f"
server_key_iv: "3ae21d4bc48d3d470eeea4f054ee1ff2"
server_key_version: "encryptionkey@0"
I20260812 01:19:25.242372 1984 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:19:25.266382 1984 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:19:25.268433 1984 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:19:25.268621 1984 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:19:25.273041 1984 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.190:46409
I20260812 01:19:25.273078 2089 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.190:46409 every 8 connection(s)
I20260812 01:19:25.273485 1984 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
I20260812 01:19:25.274510 2090 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:19:25.276921 2090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P efbfb4db2c524879a5d7d17801ff4cb0: Bootstrap starting.
I20260812 01:19:25.277482 2090 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P efbfb4db2c524879a5d7d17801ff4cb0: Neither blocks nor log segments found. Creating new log.
I20260812 01:19:25.277733 2090 log.cc:824] T 00000000000000000000000000000000 P efbfb4db2c524879a5d7d17801ff4cb0: Log is configured to *not* fsync() on all Append() calls
I20260812 01:19:25.278407 2090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P efbfb4db2c524879a5d7d17801ff4cb0: No bootstrap required, opened a new log
I20260812 01:19:25.279958 2090 raft_consensus.cc:348] T 00000000000000000000000000000000 P efbfb4db2c524879a5d7d17801ff4cb0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efbfb4db2c524879a5d7d17801ff4cb0" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 46409 } }
I20260812 01:19:25.280165 2090 raft_consensus.cc:374] T 00000000000000000000000000000000 P efbfb4db2c524879a5d7d17801ff4cb0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:19:25.280195 2090 raft_consensus.cc:729] T 00000000000000000000000000000000 P efbfb4db2c524879a5d7d17801ff4cb0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efbfb4db2c524879a5d7d17801ff4cb0, State: Initialized, Role: FOLLOWER
I20260812 01:19:25.280305 2090 consensus_queue.cc:260] T 00000000000000000000000000000000 P efbfb4db2c524879a5d7d17801ff4cb0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efbfb4db2c524879a5d7d17801ff4cb0" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 46409 } }
I20260812 01:19:25.280360 2090 raft_consensus.cc:388] T 00000000000000000000000000000000 P efbfb4db2c524879a5d7d17801ff4cb0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:19:25.280381 2090 raft_consensus.cc:482] T 00000000000000000000000000000000 P efbfb4db2c524879a5d7d17801ff4cb0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:19:25.280407 2090 raft_consensus.cc:3037] T 00000000000000000000000000000000 P efbfb4db2c524879a5d7d17801ff4cb0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:19:25.281263 2090 raft_consensus.cc:504] T 00000000000000000000000000000000 P efbfb4db2c524879a5d7d17801ff4cb0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efbfb4db2c524879a5d7d17801ff4cb0" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 46409 } }
I20260812 01:19:25.281392 2090 leader_election.cc:302] T 00000000000000000000000000000000 P efbfb4db2c524879a5d7d17801ff4cb0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: efbfb4db2c524879a5d7d17801ff4cb0; no voters:
I20260812 01:19:25.281405 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 1984
I20260812 01:19:25.281507 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/master-0/wal/instance
I20260812 01:19:25.281574 2090 leader_election.cc:288] T 00000000000000000000000000000000 P efbfb4db2c524879a5d7d17801ff4cb0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 01:19:25.281631 2095 raft_consensus.cc:2781] T 00000000000000000000000000000000 P efbfb4db2c524879a5d7d17801ff4cb0 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:19:25.281737 24578 external_mini_cluster.cc:1428] Setting key 73d161af6acaf774ce321079b536e425
I20260812 01:19:25.281785 2095 raft_consensus.cc:686] T 00000000000000000000000000000000 P efbfb4db2c524879a5d7d17801ff4cb0 [term 1 LEADER]: Becoming Leader. State: Replica: efbfb4db2c524879a5d7d17801ff4cb0, State: Running, Role: LEADER
I20260812 01:19:25.281893 2095 consensus_queue.cc:237] T 00000000000000000000000000000000 P efbfb4db2c524879a5d7d17801ff4cb0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efbfb4db2c524879a5d7d17801ff4cb0" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 46409 } }
I20260812 01:19:25.281916 2090 sys_catalog.cc:564] T 00000000000000000000000000000000 P efbfb4db2c524879a5d7d17801ff4cb0 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:19:25.282236 2097 sys_catalog.cc:455] T 00000000000000000000000000000000 P efbfb4db2c524879a5d7d17801ff4cb0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "efbfb4db2c524879a5d7d17801ff4cb0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efbfb4db2c524879a5d7d17801ff4cb0" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 46409 } } }
I20260812 01:19:25.282310 2097 sys_catalog.cc:458] T 00000000000000000000000000000000 P efbfb4db2c524879a5d7d17801ff4cb0 [sys.catalog]: This master's current role is: LEADER
I20260812 01:19:25.282507 2098 sys_catalog.cc:455] T 00000000000000000000000000000000 P efbfb4db2c524879a5d7d17801ff4cb0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader efbfb4db2c524879a5d7d17801ff4cb0. Latest consensus state: current_term: 1 leader_uuid: "efbfb4db2c524879a5d7d17801ff4cb0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efbfb4db2c524879a5d7d17801ff4cb0" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 46409 } } }
I20260812 01:19:25.282596 2098 sys_catalog.cc:458] T 00000000000000000000000000000000 P efbfb4db2c524879a5d7d17801ff4cb0 [sys.catalog]: This master's current role is: LEADER
I20260812 01:19:25.282729 2106 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:19:25.283309 2106 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:19:25.285933 2106 catalog_manager.cc:1347] Generated new cluster ID: 702944a007944b5db36839527bbbf20f
I20260812 01:19:25.285990 2106 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 01:19:25 dist-test-slave-t3q3 krb5kdc[1963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497565, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM
I20260812 01:19:25.294214 2108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:25.282406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60048 (local address 127.24.0.190:46409)
0812 01:19:25.282608 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:25.282618 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:25.282653 (+ 35us) server_negotiation.cc:407] Connection header received
0812 01:19:25.282789 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:25.282796 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:25.282852 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:25.283014 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:25.284048 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:25.285453 (+ 1405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:25.286470 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:25.287083 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:25.287213 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:25.289976 (+ 2763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:25.289980 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:25.289981 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:25.289992 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:25.292546 (+ 2554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:25.292948 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:25.292951 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:25.292951 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:25.293087 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:25.293316 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:25.293317 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:25.293317 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:25.293605 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:25.293749 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:25.293865 (+ 116us) server_negotiation.cc:299] Negotiation successful
0812 01:19:25.293993 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":72,"threads_started":1}
I20260812 01:19:25.297734 2106 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:19:25.298197 2106 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:19:25.300621 2106 catalog_manager.cc:5868] T 00000000000000000000000000000000 P efbfb4db2c524879a5d7d17801ff4cb0: Generated new TSK 0
I20260812 01:19:25.300791 2106 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.24.0.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.0.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.0.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/krb5kdc/oryx_127.24.0.129.keytab.
Entry for principal oryx/127.24.0.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/krb5kdc/oryx_127.24.0.129.keytab.
I20260812 01:19:25.331269 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.0.129:0
--local_ip_for_outbound_sockets=127.24.0.129
--webserver_interface=127.24.0.129
--webserver_port=0
--tserver_master_addrs=127.24.0.190:46409
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/krb5kdc/oryx_127.24.0.129.keytab
--principal=oryx/127.24.0.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:38987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:19:25.415176 2132 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:19:25.415421 2132 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:19:25.415460 2132 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:19:25.416622 2132 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:19:25.416695 2132 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:19:25.416741 2132 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.0.129
I20260812 01:19:25.418190 2132 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:38987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/krb5kdc/oryx_127.24.0.129.keytab
--principal=oryx/127.24.0.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.0.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.0.129
--webserver_port=0
--tserver_master_addrs=127.24.0.190:46409
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2132
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.0.129
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:19:25.418459 2132 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:19:25.418704 2132 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:19:25.423874 2146 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:25.423841 2145 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:25.423833 2149 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:19:25.423960 2132 server_base.cc:1034] running on GCE node
I20260812 01:19:25.424273 2132 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:19:25.424557 2132 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:19:25.425750 2132 hybrid_clock.cc:648] HybridClock initialized: now 1786497565425725 us; error 40 us; skew 500 ppm
Aug 12 01:19:25 dist-test-slave-t3q3 krb5kdc[1963](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497565, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.0.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:19:25.428431 2132 init.cc:377] Logged in from keytab as oryx/127.24.0.129@KRBTEST.COM (short username oryx)
I20260812 01:19:25.428891 2132 webserver.cc:460] Webserver started at http://127.24.0.129:43523/ using document root <none> and password file <none>
I20260812 01:19:25.429132 2132 fs_manager.cc:359] Metadata directory not provided
I20260812 01:19:25.429204 2132 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:19:25.429364 2132 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:19:25.430428 2132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/ts-0/data/instance:
uuid: "06e0895a4bc34d5dad8a872a6fe475eb"
format_stamp: "Formatted at 2026-08-12 01:19:25 on dist-test-slave-t3q3"
server_key: "a18da99b60fcbf0543b9a9b15666417e"
server_key_iv: "3ddd2a9371893bda27ac4002ba4eca94"
server_key_version: "encryptionkey@0"
I20260812 01:19:25.430819 2132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/ts-0/wal/instance:
uuid: "06e0895a4bc34d5dad8a872a6fe475eb"
format_stamp: "Formatted at 2026-08-12 01:19:25 on dist-test-slave-t3q3"
server_key: "a18da99b60fcbf0543b9a9b15666417e"
server_key_iv: "3ddd2a9371893bda27ac4002ba4eca94"
server_key_version: "encryptionkey@0"
I20260812 01:19:25.432251 2132 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 01:19:25.433066 2158 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:25.433269 2132 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:19:25.433378 2132 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/ts-0/wal
uuid: "06e0895a4bc34d5dad8a872a6fe475eb"
format_stamp: "Formatted at 2026-08-12 01:19:25 on dist-test-slave-t3q3"
server_key: "a18da99b60fcbf0543b9a9b15666417e"
server_key_iv: "3ddd2a9371893bda27ac4002ba4eca94"
server_key_version: "encryptionkey@0"
I20260812 01:19:25.433452 2132 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:19:25.454916 2132 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:19:25.457283 2132 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:19:25.457461 2132 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:19:25.457751 2132 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:19:25.458098 2132 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:19:25.458155 2132 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:25.458199 2132 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:19:25.458228 2132 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:25.464330 2132 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.129:34201
I20260812 01:19:25.464375 2344 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.129:34201 every 8 connection(s)
I20260812 01:19:25.464715 2132 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/ts-0/data/info.pb
I20260812 01:19:25.467083 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 2132
I20260812 01:19:25.467217 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/ts-0/wal/instance
I20260812 01:19:25.467470 24578 external_mini_cluster.cc:1428] Setting key 8ba783b14ad6952f6993839b7c4c6b54
Aug 12 01:19:25 dist-test-slave-t3q3 krb5kdc[1963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497565, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.0.129@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM
I20260812 01:19:25.475751 2108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:25.465498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.129:47671 (local address 127.24.0.190:46409)
0812 01:19:25.465571 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:25.465574 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:25.465799 (+ 225us) server_negotiation.cc:407] Connection header received
0812 01:19:25.466287 (+ 488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:25.466288 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:25.466301 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:25.466360 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:25.467510 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:25.468350 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:25.469886 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:25.470141 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:25.470228 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:25.472694 (+ 2466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:25.472696 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:25.472696 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:25.472703 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:25.474206 (+ 1503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:25.474738 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:25.474739 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:25.474740 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:25.474793 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:25.475096 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:25.475096 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:25.475097 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:25.475213 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:25.475265 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:25.475614 (+ 349us) server_negotiation.cc:299] Negotiation successful
0812 01:19:25.475684 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 01:19:25.476034 2348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:25.465481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:46409 (local address 127.24.0.129:47671)
0812 01:19:25.465662 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:25.465706 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:25.466203 (+ 497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:25.466413 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:25.466416 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:25.467099 (+ 683us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:25.467407 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:25.467415 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:25.468460 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:25.468462 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:25.469783 (+ 1321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:25.469788 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:25.470238 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:25.470240 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:25.470380 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:25.470958 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:25.470967 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:25.472592 (+ 1625us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:19:25.474294 (+ 1702us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:25.474303 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:25.474306 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:25.474650 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:25.474862 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:25.474863 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:25.474863 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:25.475019 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:25.475284 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:19:25.475291 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:19:25.475462 (+ 171us) client_negotiation.cc:769] Sending connection context
0812 01:19:25.475595 (+ 133us) client_negotiation.cc:240] Negotiation successful
0812 01:19:25.475773 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":103,"threads_started":1}
I20260812 01:19:25.476672 2346 heartbeater.cc:347] Connected to a master server at 127.24.0.190:46409
I20260812 01:19:25.476768 2346 heartbeater.cc:464] Registering TS with master...
I20260812 01:19:25.476974 2346 heartbeater.cc:511] Master 127.24.0.190:46409 requested a full tablet report, sending...
I20260812 01:19:25.477567 2035 ts_manager.cc:194] Registered new tserver with Master: 06e0895a4bc34d5dad8a872a6fe475eb (127.24.0.129:34201)
WARNING: no policy specified for oryx/127.24.0.130@KRBTEST.COM; defaulting to no policy
I20260812 01:19:25.478253 2035 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.0.129@KRBTEST.COM'} at 127.24.0.129:47671
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.0.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.0.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/krb5kdc/oryx_127.24.0.130.keytab.
Entry for principal oryx/127.24.0.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/krb5kdc/oryx_127.24.0.130.keytab.
I20260812 01:19:25.498837 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.0.130:0
--local_ip_for_outbound_sockets=127.24.0.130
--webserver_interface=127.24.0.130
--webserver_port=0
--tserver_master_addrs=127.24.0.190:46409
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/krb5kdc/oryx_127.24.0.130.keytab
--principal=oryx/127.24.0.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:38987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:19:25.582363 2355 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:19:25.582635 2355 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:19:25.582676 2355 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:19:25.583906 2355 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:19:25.583976 2355 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:19:25.584022 2355 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.0.130
I20260812 01:19:25.585562 2355 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:38987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/krb5kdc/oryx_127.24.0.130.keytab
--principal=oryx/127.24.0.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.0.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.0.130
--webserver_port=0
--tserver_master_addrs=127.24.0.190:46409
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2355
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.0.130
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:19:25.585860 2355 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:19:25.586114 2355 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:19:25.591233 2362 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:25.591214 2365 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:25.591251 2363 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:19:25.591689 2355 server_base.cc:1034] running on GCE node
I20260812 01:19:25.591884 2355 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:19:25.592204 2355 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:19:25.593358 2355 hybrid_clock.cc:648] HybridClock initialized: now 1786497565593349 us; error 29 us; skew 500 ppm
Aug 12 01:19:25 dist-test-slave-t3q3 krb5kdc[1963](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497565, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.0.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:19:25.595870 2355 init.cc:377] Logged in from keytab as oryx/127.24.0.130@KRBTEST.COM (short username oryx)
I20260812 01:19:25.596390 2355 webserver.cc:460] Webserver started at http://127.24.0.130:36465/ using document root <none> and password file <none>
I20260812 01:19:25.596644 2355 fs_manager.cc:359] Metadata directory not provided
I20260812 01:19:25.596717 2355 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:19:25.596873 2355 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:19:25.597963 2355 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/ts-1/data/instance:
uuid: "d645f2e569134a37baea2860781c5e0e"
format_stamp: "Formatted at 2026-08-12 01:19:25 on dist-test-slave-t3q3"
server_key: "d04f40c4310226eea803ec7ea8dc7df1"
server_key_iv: "199e0cdc11be2afe4ca45f67eb1d48a7"
server_key_version: "encryptionkey@0"
I20260812 01:19:25.598351 2355 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/ts-1/wal/instance:
uuid: "d645f2e569134a37baea2860781c5e0e"
format_stamp: "Formatted at 2026-08-12 01:19:25 on dist-test-slave-t3q3"
server_key: "d04f40c4310226eea803ec7ea8dc7df1"
server_key_iv: "199e0cdc11be2afe4ca45f67eb1d48a7"
server_key_version: "encryptionkey@0"
I20260812 01:19:25.599826 2355 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 01:19:25.600780 2379 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:25.600957 2355 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 01:19:25.601101 2355 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/ts-1/wal
uuid: "d645f2e569134a37baea2860781c5e0e"
format_stamp: "Formatted at 2026-08-12 01:19:25 on dist-test-slave-t3q3"
server_key: "d04f40c4310226eea803ec7ea8dc7df1"
server_key_iv: "199e0cdc11be2afe4ca45f67eb1d48a7"
server_key_version: "encryptionkey@0"
I20260812 01:19:25.601186 2355 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:19:25.608129 2355 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:19:25.609966 2355 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:19:25.610191 2355 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:19:25.610423 2355 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:19:25.610745 2355 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:19:25.610801 2355 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:25.610843 2355 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:19:25.610869 2355 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:25.617137 2355 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.130:45783
I20260812 01:19:25.617218 2579 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.130:45783 every 8 connection(s)
I20260812 01:19:25.617580 2355 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/ts-1/data/info.pb
Aug 12 01:19:25 dist-test-slave-t3q3 krb5kdc[1963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497565, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.0.130@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM
I20260812 01:19:25.624555 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 2355
I20260812 01:19:25.624706 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/ts-1/wal/instance
I20260812 01:19:25.624961 24578 external_mini_cluster.cc:1428] Setting key fa656aee1b280cc48229c65482f657db
I20260812 01:19:25.627524 2108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:25.618417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.130:48759 (local address 127.24.0.190:46409)
0812 01:19:25.618468 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:25.618470 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:25.618670 (+ 200us) server_negotiation.cc:407] Connection header received
0812 01:19:25.619192 (+ 522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:25.619193 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:25.619203 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:25.619266 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:25.620264 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:25.620788 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:25.621832 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:25.622015 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:25.622069 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:25.624260 (+ 2191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:25.624262 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:25.624262 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:25.624269 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:25.625989 (+ 1720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:25.626520 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:25.626521 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:25.626521 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:25.626588 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:25.626895 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:25.626895 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:25.626895 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:25.627002 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:25.627058 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:25.627391 (+ 333us) server_negotiation.cc:299] Negotiation successful
0812 01:19:25.627461 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 01:19:25.627856 2582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:25.618381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:46409 (local address 127.24.0.130:48759)
0812 01:19:25.618538 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:25.618582 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:25.619093 (+ 511us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:25.619321 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:25.619324 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:25.619912 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:25.620163 (+ 251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:25.620169 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:25.620888 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:25.620888 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:25.621748 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:25.621751 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:25.622114 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:25.622115 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:25.622237 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:25.622721 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:25.622730 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:25.624172 (+ 1442us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:19:25.626074 (+ 1902us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:25.626083 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:25.626086 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:25.626423 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:25.626668 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:25.626668 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:25.626669 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:25.626823 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:25.627071 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:19:25.627078 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:19:25.627247 (+ 169us) client_negotiation.cc:769] Sending connection context
0812 01:19:25.627366 (+ 119us) client_negotiation.cc:240] Negotiation successful
0812 01:19:25.627555 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":82,"threads_started":1}
I20260812 01:19:25.628422 2580 heartbeater.cc:347] Connected to a master server at 127.24.0.190:46409
I20260812 01:19:25.628537 2580 heartbeater.cc:464] Registering TS with master...
I20260812 01:19:25.628743 2580 heartbeater.cc:511] Master 127.24.0.190:46409 requested a full tablet report, sending...
I20260812 01:19:25.629159 2035 ts_manager.cc:194] Registered new tserver with Master: d645f2e569134a37baea2860781c5e0e (127.24.0.130:45783)
I20260812 01:19:25.629549 2035 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.0.130@KRBTEST.COM'} at 127.24.0.130:48759
WARNING: no policy specified for oryx/127.24.0.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.0.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.0.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/krb5kdc/oryx_127.24.0.131.keytab.
Entry for principal oryx/127.24.0.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/krb5kdc/oryx_127.24.0.131.keytab.
I20260812 01:19:25.654531 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.0.131:0
--local_ip_for_outbound_sockets=127.24.0.131
--webserver_interface=127.24.0.131
--webserver_port=0
--tserver_master_addrs=127.24.0.190:46409
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/krb5kdc/oryx_127.24.0.131.keytab
--principal=oryx/127.24.0.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:38987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:19:25.739646 2585 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:19:25.739910 2585 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:19:25.739950 2585 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:19:25.741155 2585 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:19:25.741226 2585 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:19:25.741271 2585 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.0.131
I20260812 01:19:25.742758 2585 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:38987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/krb5kdc/oryx_127.24.0.131.keytab
--principal=oryx/127.24.0.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.0.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.0.131
--webserver_port=0
--tserver_master_addrs=127.24.0.190:46409
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2585
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.0.131
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:19:25.743055 2585 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:19:25.743299 2585 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 01:19:25.748831 2585 server_base.cc:1034] running on GCE node
W20260812 01:19:25.748813 2592 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:25.748812 2594 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:25.748807 2590 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:19:25.749246 2585 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:19:25.749537 2585 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:19:25.750708 2585 hybrid_clock.cc:648] HybridClock initialized: now 1786497565750691 us; error 47 us; skew 500 ppm
Aug 12 01:19:25 dist-test-slave-t3q3 krb5kdc[1963](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497565, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.0.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:19:25.753326 2585 init.cc:377] Logged in from keytab as oryx/127.24.0.131@KRBTEST.COM (short username oryx)
I20260812 01:19:25.753813 2585 webserver.cc:460] Webserver started at http://127.24.0.131:43941/ using document root <none> and password file <none>
I20260812 01:19:25.754060 2585 fs_manager.cc:359] Metadata directory not provided
I20260812 01:19:25.754140 2585 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:19:25.754295 2585 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:19:25.755368 2585 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/ts-2/data/instance:
uuid: "85a369874676488a8c43c1a2cf51a95a"
format_stamp: "Formatted at 2026-08-12 01:19:25 on dist-test-slave-t3q3"
server_key: "076ef42558e5affa82a5a770341874dd"
server_key_iv: "f052eb34a4dcf94de07288d3f4483747"
server_key_version: "encryptionkey@0"
I20260812 01:19:25.755743 2585 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/ts-2/wal/instance:
uuid: "85a369874676488a8c43c1a2cf51a95a"
format_stamp: "Formatted at 2026-08-12 01:19:25 on dist-test-slave-t3q3"
server_key: "076ef42558e5affa82a5a770341874dd"
server_key_iv: "f052eb34a4dcf94de07288d3f4483747"
server_key_version: "encryptionkey@0"
I20260812 01:19:25.757201 2585 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 01:19:25.757972 2605 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:25.758163 2585 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:25.758271 2585 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/ts-2/wal
uuid: "85a369874676488a8c43c1a2cf51a95a"
format_stamp: "Formatted at 2026-08-12 01:19:25 on dist-test-slave-t3q3"
server_key: "076ef42558e5affa82a5a770341874dd"
server_key_iv: "f052eb34a4dcf94de07288d3f4483747"
server_key_version: "encryptionkey@0"
I20260812 01:19:25.758368 2585 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:19:25.768730 2585 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:19:25.770661 2585 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:19:25.770824 2585 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:19:25.771051 2585 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:19:25.771382 2585 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:19:25.771437 2585 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:25.771478 2585 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:19:25.771502 2585 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:25.777631 2585 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.131:41837
I20260812 01:19:25.777683 2782 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.131:41837 every 8 connection(s)
I20260812 01:19:25.778049 2585 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/ts-2/data/info.pb
I20260812 01:19:25.780260 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 2585
I20260812 01:19:25.780382 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/ts-2/wal/instance
I20260812 01:19:25.780571 24578 external_mini_cluster.cc:1428] Setting key 2d44de0f72cf85d0a88f8d5a1e325ef7
Aug 12 01:19:25 dist-test-slave-t3q3 krb5kdc[1963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497565, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.0.131@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM
I20260812 01:19:25.790325 2108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:25.778872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.131:40205 (local address 127.24.0.190:46409)
0812 01:19:25.778941 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:25.778943 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:25.779165 (+ 222us) server_negotiation.cc:407] Connection header received
0812 01:19:25.779640 (+ 475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:25.779641 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:25.779651 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:25.779713 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:25.780880 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:25.781616 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:25.783266 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:25.783473 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:25.783552 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:25.786583 (+ 3031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:25.786585 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:25.786586 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:25.786595 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:25.788353 (+ 1758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:25.788943 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:25.788944 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:25.788945 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:25.789019 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:25.789477 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:25.789478 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:25.789479 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:25.789620 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:25.789714 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:25.790128 (+ 414us) server_negotiation.cc:299] Negotiation successful
0812 01:19:25.790230 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 01:19:25.790570 2787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:25.778851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:46409 (local address 127.24.0.131:40205)
0812 01:19:25.779025 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:25.779069 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:25.779554 (+ 485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:25.779765 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:25.779769 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:25.780527 (+ 758us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:25.780777 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:25.780784 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:25.781739 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:25.781741 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:25.783176 (+ 1435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:25.783180 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:25.783559 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:25.783560 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:25.783719 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:25.784412 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:25.784422 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:25.786459 (+ 2037us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:19:25.788434 (+ 1975us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:25.788443 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:25.788445 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:25.788792 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:25.789244 (+ 452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:25.789245 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:25.789246 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:25.789380 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:25.789748 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:19:25.789756 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:19:25.789925 (+ 169us) client_negotiation.cc:769] Sending connection context
0812 01:19:25.790042 (+ 117us) client_negotiation.cc:240] Negotiation successful
0812 01:19:25.790284 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":98,"threads_started":1}
I20260812 01:19:25.791225 2785 heartbeater.cc:347] Connected to a master server at 127.24.0.190:46409
I20260812 01:19:25.791335 2785 heartbeater.cc:464] Registering TS with master...
I20260812 01:19:25.791523 2785 heartbeater.cc:511] Master 127.24.0.190:46409 requested a full tablet report, sending...
I20260812 01:19:25.791929 2035 ts_manager.cc:194] Registered new tserver with Master: 85a369874676488a8c43c1a2cf51a95a (127.24.0.131:41837)
I20260812 01:19:25.792340 2035 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.0.131@KRBTEST.COM'} at 127.24.0.131:40205
I20260812 01:19:25.793179 24578 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.24.0.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.0.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.0.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/krb5kdc/oryx_127.24.0.189.keytab.
Entry for principal oryx/127.24.0.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/krb5kdc/oryx_127.24.0.189.keytab.
I20260812 01:19:25.823567 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.189:38795
--webserver_interface=127.24.0.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/krb5kdc/oryx_127.24.0.189.keytab
--principal=oryx/127.24.0.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:38987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.0.190:46409,127.24.0.189:38795
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:19:25.907442 2791 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:19:25.907680 2791 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:19:25.907719 2791 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:19:25.908919 2791 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:19:25.908988 2791 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:19:25.909013 2791 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:19:25.909044 2791 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:19:25.909075 2791 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:19:25.910571 2791 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:38987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.24.0.190:46409,127.24.0.189:38795
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/krb5kdc/oryx_127.24.0.189.keytab
--principal=oryx/127.24.0.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.189:38795
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.24.0.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2791
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
W20260812 01:19:25.910722 2791 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.24.0.190:46409,127.24.0.189:38795'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260812 01:19:25.910879 2791 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:19:25.911104 2791 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:19:25.916302 2801 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:25.916299 2805 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:25.916316 2803 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:19:25.916594 2791 server_base.cc:1034] running on GCE node
I20260812 01:19:25.916859 2791 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:19:25.917147 2791 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:19:25.918320 2791 hybrid_clock.cc:648] HybridClock initialized: now 1786497565918302 us; error 41 us; skew 500 ppm
Aug 12 01:19:25 dist-test-slave-t3q3 krb5kdc[1963](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497565, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.0.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:19:25.920944 2791 init.cc:377] Logged in from keytab as oryx/127.24.0.189@KRBTEST.COM (short username oryx)
I20260812 01:19:25.921427 2791 webserver.cc:460] Webserver started at http://127.24.0.189:41623/ using document root <none> and password file <none>
I20260812 01:19:25.921645 2791 fs_manager.cc:359] Metadata directory not provided
I20260812 01:19:25.921716 2791 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:19:25.921870 2791 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:19:25.922890 2791 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/master-1/data/instance:
uuid: "35a50d99aa1b4d9c8a4b11849067ad0f"
format_stamp: "Formatted at 2026-08-12 01:19:25 on dist-test-slave-t3q3"
server_key: "239706af31e5d548ba29a94f363070f4"
server_key_iv: "0ac14dd942d48ae7dba426580778666e"
server_key_version: "encryptionkey@0"
I20260812 01:19:25.923260 2791 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/master-1/wal/instance:
uuid: "35a50d99aa1b4d9c8a4b11849067ad0f"
format_stamp: "Formatted at 2026-08-12 01:19:25 on dist-test-slave-t3q3"
server_key: "239706af31e5d548ba29a94f363070f4"
server_key_iv: "0ac14dd942d48ae7dba426580778666e"
server_key_version: "encryptionkey@0"
I20260812 01:19:25.924686 2791 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260812 01:19:25.925477 2813 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:25.925676 2791 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:19:25.925781 2791 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/master-1/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/master-1/wal
uuid: "35a50d99aa1b4d9c8a4b11849067ad0f"
format_stamp: "Formatted at 2026-08-12 01:19:25 on dist-test-slave-t3q3"
server_key: "239706af31e5d548ba29a94f363070f4"
server_key_iv: "0ac14dd942d48ae7dba426580778666e"
server_key_version: "encryptionkey@0"
I20260812 01:19:25.925856 2791 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:19:25.937673 2791 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:19:25.939898 2791 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:19:25.940102 2791 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:19:25.944720 2791 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.189:38795
I20260812 01:19:25.944777 2909 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.189:38795 every 8 connection(s)
I20260812 01:19:25.945346 2791 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/master-1/data/info.pb
I20260812 01:19:25.946475 2910 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:19:25.947988 2910 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.0.190" port: 46409 } has no permanent_uuid. Determining permanent_uuid...
I20260812 01:19:25.949431 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 2791
Aug 12 01:19:25 dist-test-slave-t3q3 krb5kdc[1963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497565, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.0.189@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM
Aug 12 01:19:25 dist-test-slave-t3q3 krb5kdc[1963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497565, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.0.189@KRBTEST.COM
I20260812 01:19:25.958762 2108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:25.948756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60064 (local address 127.24.0.190:46409)
0812 01:19:25.948844 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:25.948847 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:25.949073 (+ 226us) server_negotiation.cc:407] Connection header received
0812 01:19:25.949577 (+ 504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:25.949579 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:25.949593 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:25.949649 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:25.950566 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:25.951281 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:25.952706 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:25.952907 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:25.953011 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:25.955600 (+ 2589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:25.955601 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:25.955602 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:25.955609 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:25.957208 (+ 1599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:25.957646 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:25.957648 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:25.957649 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:25.957722 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:25.957956 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:25.957957 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:25.957957 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:25.958199 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:25.958289 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:25.958579 (+ 290us) server_negotiation.cc:299] Negotiation successful
0812 01:19:25.958668 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 01:19:25.958963 2912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:25.948726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:46409 (local address 127.0.0.1:60064)
0812 01:19:25.948954 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:25.948989 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:25.949462 (+ 473us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:25.949722 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:25.949727 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:25.950157 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:25.950450 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:25.950455 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:25.951395 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:25.951396 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:25.952610 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:25.952613 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:25.953025 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:25.953026 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:25.953123 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:25.953647 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:25.953655 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:25.955503 (+ 1848us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:19:25.957271 (+ 1768us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:25.957278 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:25.957281 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:25.957566 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:25.957798 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:25.957798 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:25.957798 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:25.957887 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:25.958306 (+ 419us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:19:25.958312 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:19:25.958436 (+ 124us) client_negotiation.cc:769] Sending connection context
0812 01:19:25.958551 (+ 115us) client_negotiation.cc:240] Negotiation successful
0812 01:19:25.958717 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"mutex_wait_us":204,"thread_start_us":137,"threads_started":1}
I20260812 01:19:25.959615 2910 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.0.189" port: 38795 } has no permanent_uuid. Determining permanent_uuid...
I20260812 01:19:25.959843 2917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:25.949992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44070 (local address 127.24.0.189:38795)
0812 01:19:25.950173 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:25.950177 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:25.950226 (+ 49us) server_negotiation.cc:407] Connection header received
0812 01:19:25.950271 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:25.950273 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:25.950293 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:25.950390 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:25.950943 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:25.951991 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:25.952963 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:25.953335 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:25.953428 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:25.956288 (+ 2860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:25.956295 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:25.956295 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:25.956301 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:25.958351 (+ 2050us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:25.958749 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:25.958752 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:25.958753 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:25.958822 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:25.959150 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:25.959151 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:25.959151 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:25.959331 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:25.959486 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:25.959561 (+ 75us) server_negotiation.cc:299] Negotiation successful
0812 01:19:25.959643 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":121,"threads_started":1}
Aug 12 01:19:25 dist-test-slave-t3q3 krb5kdc[1963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497565, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.0.189@KRBTEST.COM for oryx/127.24.0.189@KRBTEST.COM
I20260812 01:19:25.967209 2912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:25.959893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.189:38795 (local address 127.0.0.1:44078)
0812 01:19:25.959938 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:25.959946 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:25.959982 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:25.960359 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:25.960360 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:25.960470 (+ 110us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:25.960519 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:25.960521 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:25.961322 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:25.961323 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:25.961799 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:25.961800 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:25.962223 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:25.962224 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:25.962285 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:25.962703 (+ 418us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:25.962708 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:25.963868 (+ 1160us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:19:25.966091 (+ 2223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:25.966092 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:25.966092 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:25.966264 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:25.966499 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:25.966499 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:25.966500 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:25.966562 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:25.966826 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:19:25.966827 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:19:25.966869 (+ 42us) client_negotiation.cc:769] Sending connection context
0812 01:19:25.966962 (+ 93us) client_negotiation.cc:240] Negotiation successful
0812 01:19:25.967067 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260812 01:19:25.967306 2919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:25.959943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44078 (local address 127.24.0.189:38795)
0812 01:19:25.960208 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:25.960211 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:25.960219 (+ 8us) server_negotiation.cc:407] Connection header received
0812 01:19:25.960240 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:25.960241 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:25.960252 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:25.960342 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:25.960600 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:25.961251 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:25.961869 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:25.962112 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:25.962192 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:25.963946 (+ 1754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:25.963947 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:25.963947 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:25.963953 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:25.965993 (+ 2040us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:25.966357 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:25.966359 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:25.966359 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:25.966419 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:25.966635 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:25.966636 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:25.966636 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:25.966753 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:25.966835 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:25.966987 (+ 152us) server_negotiation.cc:299] Negotiation successful
0812 01:19:25.967145 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":82,"server-negotiator.queue_time_us":243,"thread_start_us":231,"threads_started":1}
I20260812 01:19:25.968688 2910 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 35a50d99aa1b4d9c8a4b11849067ad0f: Bootstrap starting.
I20260812 01:19:25.969385 2910 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 35a50d99aa1b4d9c8a4b11849067ad0f: Neither blocks nor log segments found. Creating new log.
I20260812 01:19:25.969703 2910 log.cc:824] T 00000000000000000000000000000000 P 35a50d99aa1b4d9c8a4b11849067ad0f: Log is configured to *not* fsync() on all Append() calls
I20260812 01:19:25.970455 2910 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 35a50d99aa1b4d9c8a4b11849067ad0f: No bootstrap required, opened a new log
I20260812 01:19:25.971943 2910 raft_consensus.cc:348] T 00000000000000000000000000000000 P 35a50d99aa1b4d9c8a4b11849067ad0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efbfb4db2c524879a5d7d17801ff4cb0" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 46409 } } peers { permanent_uuid: "35a50d99aa1b4d9c8a4b11849067ad0f" member_type: VOTER last_known_addr { host: "127.24.0.189" port: 38795 } }
I20260812 01:19:25.972127 2910 raft_consensus.cc:374] T 00000000000000000000000000000000 P 35a50d99aa1b4d9c8a4b11849067ad0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:19:25.972185 2910 raft_consensus.cc:729] T 00000000000000000000000000000000 P 35a50d99aa1b4d9c8a4b11849067ad0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35a50d99aa1b4d9c8a4b11849067ad0f, State: Initialized, Role: FOLLOWER
I20260812 01:19:25.972321 2910 consensus_queue.cc:260] T 00000000000000000000000000000000 P 35a50d99aa1b4d9c8a4b11849067ad0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efbfb4db2c524879a5d7d17801ff4cb0" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 46409 } } peers { permanent_uuid: "35a50d99aa1b4d9c8a4b11849067ad0f" member_type: VOTER last_known_addr { host: "127.24.0.189" port: 38795 } }
I20260812 01:19:25.972569 2923 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35a50d99aa1b4d9c8a4b11849067ad0f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efbfb4db2c524879a5d7d17801ff4cb0" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 46409 } } peers { permanent_uuid: "35a50d99aa1b4d9c8a4b11849067ad0f" member_type: VOTER last_known_addr { host: "127.24.0.189" port: 38795 } } }
I20260812 01:19:25.972641 2910 sys_catalog.cc:564] T 00000000000000000000000000000000 P 35a50d99aa1b4d9c8a4b11849067ad0f [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:19:25.972651 2923 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35a50d99aa1b4d9c8a4b11849067ad0f [sys.catalog]: This master's current role is: FOLLOWER
W20260812 01:19:25.974478 2937 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 35a50d99aa1b4d9c8a4b11849067ad0f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 01:19:25.974542 2937 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 01:19:25.976316 2791 master_runner.cc:428] Detected that this master 35a50d99aa1b4d9c8a4b11849067ad0f is joining an existing cluster
I20260812 01:19:25.976372 2791 master_runner.cc:432] Initiating AddMaster RPC to add 127.24.0.189:38795
I20260812 01:19:25.984802 2945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:25.977733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.189:38795 (local address 127.0.0.1:44088)
0812 01:19:25.977877 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:25.977885 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:25.977934 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:25.978054 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:25.978054 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:25.978266 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:25.978329 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:25.978330 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:25.979229 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:25.979230 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:25.980297 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:25.980300 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:25.980819 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:25.980821 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:25.980891 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:25.981279 (+ 388us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:25.981285 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:25.981743 (+ 458us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:19:25.983661 (+ 1918us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:25.983662 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:25.983662 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:25.983911 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:25.984167 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:25.984167 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:25.984168 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:25.984227 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:25.984468 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:19:25.984468 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:19:25.984512 (+ 44us) client_negotiation.cc:769] Sending connection context
0812 01:19:25.984588 (+ 76us) client_negotiation.cc:240] Negotiation successful
0812 01:19:25.984697 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"mutex_wait_us":29,"thread_start_us":74,"threads_started":1}
I20260812 01:19:25.985001 2108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:25.977609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60072 (local address 127.24.0.190:46409)
0812 01:19:25.977665 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:25.977667 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:25.977761 (+ 94us) server_negotiation.cc:407] Connection header received
0812 01:19:25.977880 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:25.977881 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:25.977894 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:25.977963 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:25.978469 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:25.979192 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:25.979996 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:25.980288 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:25.980361 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:25.981615 (+ 1254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:25.981617 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:25.981618 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:25.981625 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:25.983762 (+ 2137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:25.984116 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:25.984118 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:25.984118 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:25.984187 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:25.984395 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:25.984396 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:25.984396 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:25.984531 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:25.984605 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:25.984821 (+ 216us) server_negotiation.cc:299] Negotiation successful
0812 01:19:25.984910 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260812 01:19:25.985384 2943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:25.977510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:46409 (local address 127.0.0.1:60072)
0812 01:19:25.977699 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:25.977709 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:25.977788 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:25.978014 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:25.978015 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:25.978314 (+ 299us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:25.978394 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:25.978396 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:25.979274 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:25.979275 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:25.979896 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:25.979898 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:25.980385 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:25.980386 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:25.980453 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:25.981057 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:25.981062 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:25.981531 (+ 469us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:19:25.983839 (+ 2308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:25.983839 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:25.983840 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:25.984015 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:25.984259 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:25.984259 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:25.984260 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:25.984320 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:25.984617 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:19:25.984618 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:19:25.984682 (+ 64us) client_negotiation.cc:769] Sending connection context
0812 01:19:25.984779 (+ 97us) client_negotiation.cc:240] Negotiation successful
0812 01:19:25.985268 (+ 489us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"mutex_wait_us":40,"thread_start_us":80,"threads_started":1}
I20260812 01:19:25.985683 2919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:25.977794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44088 (local address 127.24.0.189:38795)
0812 01:19:25.977830 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:25.977832 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:25.977925 (+ 93us) server_negotiation.cc:407] Connection header received
0812 01:19:25.977987 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:25.977987 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:25.977997 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:25.978048 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:25.978416 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:25.979133 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:25.980402 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:25.980737 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:25.980817 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:25.981837 (+ 1020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:25.981838 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:25.981838 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:25.981845 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:25.983570 (+ 1725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:25.984002 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:25.984003 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:25.984003 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:25.984055 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:25.984292 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:25.984292 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:25.984293 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:25.984410 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:25.984479 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:25.985510 (+ 1031us) server_negotiation.cc:299] Negotiation successful
0812 01:19:25.985596 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
Aug 12 01:19:25 dist-test-slave-t3q3 krb5kdc[1963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497565, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.0.190@KRBTEST.COM for oryx/127.24.0.189@KRBTEST.COM
I20260812 01:19:25.995201 2919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:25.987677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44104 (local address 127.24.0.189:38795)
0812 01:19:25.987711 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:25.987713 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:25.987899 (+ 186us) server_negotiation.cc:407] Connection header received
0812 01:19:25.987989 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:25.987990 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:25.988001 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:25.988104 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:25.988496 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:25.989276 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:25.989993 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:25.990314 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:25.990383 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:25.992706 (+ 2323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:25.992707 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:25.992707 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:25.992713 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:25.994016 (+ 1303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:25.994481 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:25.994482 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:25.994482 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:25.994530 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:25.994758 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:25.994758 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:25.994758 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:25.994847 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:25.994896 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:25.995096 (+ 200us) server_negotiation.cc:299] Negotiation successful
0812 01:19:25.995148 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 01:19:25.995319 2947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:25.987618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.189:38795 (local address 127.0.0.1:44104)
0812 01:19:25.987773 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:25.987821 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:25.987901 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:25.988106 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:25.988109 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:25.988372 (+ 263us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:25.988416 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:25.988418 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:25.989368 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:25.989369 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:25.989914 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:25.989916 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:25.990399 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:25.990400 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:25.990453 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:25.990896 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:25.990901 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:25.992619 (+ 1718us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:19:25.994104 (+ 1485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:25.994111 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:25.994114 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:25.994404 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:25.994603 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:25.994604 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:25.994605 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:25.994683 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:25.994913 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:19:25.994920 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:19:25.994977 (+ 57us) client_negotiation.cc:769] Sending connection context
0812 01:19:25.995082 (+ 105us) client_negotiation.cc:240] Negotiation successful
0812 01:19:25.995208 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":105,"threads_started":1}
I20260812 01:19:25.995986 2035 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.24.0.189:38795: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "35a50d99aa1b4d9c8a4b11849067ad0f"
member_type: NON_VOTER
last_known_addr {
host: "127.24.0.189"
port: 38795
}
attrs {
promote: true
}
}
dest_uuid: "efbfb4db2c524879a5d7d17801ff4cb0"
cas_config_opid_index: -1
I20260812 01:19:25.996369 2035 consensus_queue.cc:237] T 00000000000000000000000000000000 P efbfb4db2c524879a5d7d17801ff4cb0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "efbfb4db2c524879a5d7d17801ff4cb0" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 46409 } } peers { permanent_uuid: "35a50d99aa1b4d9c8a4b11849067ad0f" member_type: NON_VOTER last_known_addr { host: "127.24.0.189" port: 38795 } attrs { promote: true } }
I20260812 01:19:25.996906 2951 raft_consensus.cc:2932] T 00000000000000000000000000000000 P efbfb4db2c524879a5d7d17801ff4cb0 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 35a50d99aa1b4d9c8a4b11849067ad0f (127.24.0.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "efbfb4db2c524879a5d7d17801ff4cb0" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 46409 } } peers { permanent_uuid: "35a50d99aa1b4d9c8a4b11849067ad0f" member_type: NON_VOTER last_known_addr { host: "127.24.0.189" port: 38795 } attrs { promote: true } } }
I20260812 01:19:25.997196 2856 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 35a50d99aa1b4d9c8a4b11849067ad0f [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:19:25.998059 2951 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.24.0.189:38795
I20260812 01:19:25.998256 2856 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 35a50d99aa1b4d9c8a4b11849067ad0f [term 1 FOLLOWER]: Refusing update from remote peer efbfb4db2c524879a5d7d17801ff4cb0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260812 01:19:25.998327 2950 sys_catalog.cc:455] T 00000000000000000000000000000000 P efbfb4db2c524879a5d7d17801ff4cb0 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "efbfb4db2c524879a5d7d17801ff4cb0" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "efbfb4db2c524879a5d7d17801ff4cb0" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 46409 } } peers { permanent_uuid: "35a50d99aa1b4d9c8a4b11849067ad0f" member_type: NON_VOTER last_known_addr { host: "127.24.0.189" port: 38795 } attrs { promote: true } } }
I20260812 01:19:25.998370 2791 master.cc:537] Master@127.24.0.189:38795 shutting down...
I20260812 01:19:25.998400 2950 sys_catalog.cc:458] T 00000000000000000000000000000000 P efbfb4db2c524879a5d7d17801ff4cb0 [sys.catalog]: This master's current role is: LEADER
I20260812 01:19:25.998556 2950 consensus_queue.cc:1035] T 00000000000000000000000000000000 P efbfb4db2c524879a5d7d17801ff4cb0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35a50d99aa1b4d9c8a4b11849067ad0f" member_type: NON_VOTER last_known_addr { host: "127.24.0.189" port: 38795 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 01:19:25.999189 2009 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260812 01:19:25.999599 2009 consensus_peers.cc:487] T 00000000000000000000000000000000 P efbfb4db2c524879a5d7d17801ff4cb0 -> Peer 35a50d99aa1b4d9c8a4b11849067ad0f (127.24.0.189:38795): Couldn't send request to peer 35a50d99aa1b4d9c8a4b11849067ad0f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260812 01:19:26.001621 2791 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 35a50d99aa1b4d9c8a4b11849067ad0f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 01:19:26.001718 2791 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 35a50d99aa1b4d9c8a4b11849067ad0f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:19:26.001773 2791 tablet_replica.cc:330] T 00000000000000000000000000000000 P 35a50d99aa1b4d9c8a4b11849067ad0f: stopping tablet replica
I20260812 01:19:26.014334 2791 master.cc:552] Master@127.24.0.189:38795 shutdown complete.
I20260812 01:19:26.014401 2791 master_runner.cc:305] Clearing existing system tablet
I20260812 01:19:26.014531 2791 fs_manager.cc:359] Metadata directory not provided
I20260812 01:19:26.014595 2791 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:19:26.015246 2791 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260812 01:19:26.015763 2957 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:26.015929 2791 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:19:26.016009 2791 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:26.016039 2791 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/master-1/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/master-1/wal
uuid: "35a50d99aa1b4d9c8a4b11849067ad0f"
format_stamp: "Formatted at 2026-08-12 01:19:25 on dist-test-slave-t3q3"
server_key: "239706af31e5d548ba29a94f363070f4"
server_key_iv: "0ac14dd942d48ae7dba426580778666e"
server_key_version: "encryptionkey@0"
I20260812 01:19:26.016999 2791 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 35a50d99aa1b4d9c8a4b11849067ad0f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:19:26.018567 2791 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 35a50d99aa1b4d9c8a4b11849067ad0f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260812 01:19:26.018659 2791 log.cc:1196] T 00000000000000000000000000000000 P 35a50d99aa1b4d9c8a4b11849067ad0f: Deleting WAL directory at /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260812 01:19:26.018894 2791 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 35a50d99aa1b4d9c8a4b11849067ad0f: Deleting consensus metadata
I20260812 01:19:26.019039 2791 master_runner.cc:315] Copying system tablet from 127.24.0.190:46409
I20260812 01:19:26.019737 2791 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 35a50d99aa1b4d9c8a4b11849067ad0f: tablet copy: Beginning tablet copy session from remote peer at address 127.24.0.190:46409
I20260812 01:19:26.027184 2108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:26.020198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60088 (local address 127.24.0.190:46409)
0812 01:19:26.020298 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:26.020300 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:26.020338 (+ 38us) server_negotiation.cc:407] Connection header received
0812 01:19:26.020426 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:26.020427 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:26.020436 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:26.020487 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:26.020933 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.021423 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.023111 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.023296 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.023348 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:26.024487 (+ 1139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:26.024489 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:26.024489 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:26.024495 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:26.025971 (+ 1476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:26.026398 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:26.026399 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:26.026400 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:26.026454 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:26.026669 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:26.026669 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:26.026670 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:26.026771 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:26.026823 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:26.027048 (+ 225us) server_negotiation.cc:299] Negotiation successful
0812 01:19:26.027115 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260812 01:19:26.027243 2965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:26.020012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:46409 (local address 127.0.0.1:60088)
0812 01:19:26.020267 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:26.020280 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:26.020357 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:26.020495 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:26.020495 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:26.020773 (+ 278us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:26.020849 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:26.020851 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.021507 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.021508 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:26.023022 (+ 1514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:26.023025 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.023374 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.023375 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:26.023453 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:26.023927 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:26.023933 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:26.024401 (+ 468us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:19:26.026060 (+ 1659us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:26.026061 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:26.026062 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:26.026313 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:26.026520 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:26.026521 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:26.026521 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:26.026601 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:26.026837 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:19:26.026838 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:19:26.026926 (+ 88us) client_negotiation.cc:769] Sending connection context
0812 01:19:26.027004 (+ 78us) client_negotiation.cc:240] Negotiation successful
0812 01:19:26.027125 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":196,"threads_started":1}
I20260812 01:19:26.027712 2066 tablet_copy_service.cc:140] P efbfb4db2c524879a5d7d17801ff4cb0: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 35a50d99aa1b4d9c8a4b11849067ad0f ({username='oryx', principal='oryx/127.24.0.189@KRBTEST.COM'} at 127.0.0.1:60088)
I20260812 01:19:26.027850 2066 tablet_copy_service.cc:161] P efbfb4db2c524879a5d7d17801ff4cb0: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 35a50d99aa1b4d9c8a4b11849067ad0f at {username='oryx', principal='oryx/127.24.0.189@KRBTEST.COM'} at 127.0.0.1:60088: session id = 35a50d99aa1b4d9c8a4b11849067ad0f-00000000000000000000000000000000
I20260812 01:19:26.028648 2066 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P efbfb4db2c524879a5d7d17801ff4cb0: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 01:19:26.029285 2791 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260812 01:19:26.030735 2791 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 35a50d99aa1b4d9c8a4b11849067ad0f: tablet copy: Starting download of 0 data blocks...
I20260812 01:19:26.030903 2791 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 35a50d99aa1b4d9c8a4b11849067ad0f: tablet copy: Starting download of 1 WAL segments...
I20260812 01:19:26.032049 2791 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 35a50d99aa1b4d9c8a4b11849067ad0f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 01:19:26.033066 2066 tablet_copy_service.cc:342] P efbfb4db2c524879a5d7d17801ff4cb0: Request end of tablet copy session 35a50d99aa1b4d9c8a4b11849067ad0f-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.24.0.189@KRBTEST.COM'} at 127.0.0.1:60088
I20260812 01:19:26.033133 2066 tablet_copy_service.cc:433] P efbfb4db2c524879a5d7d17801ff4cb0: ending tablet copy session 35a50d99aa1b4d9c8a4b11849067ad0f-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 35a50d99aa1b4d9c8a4b11849067ad0f
I20260812 01:19:26.035699 2791 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:19:26.036955 2968 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:26.037037 2979 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:26.037109 2977 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:19:26.037410 2791 server_base.cc:1034] running on GCE node
I20260812 01:19:26.037523 2791 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:19:26.037681 2791 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:19:26.038836 2791 hybrid_clock.cc:648] HybridClock initialized: now 1786497566038826 us; error 24 us; skew 500 ppm
Aug 12 01:19:26 dist-test-slave-t3q3 krb5kdc[1963](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497566, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.0.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:19:26.040241 2791 init.cc:377] Logged in from keytab as oryx/127.24.0.189@KRBTEST.COM (short username oryx)
I20260812 01:19:26.040549 2791 webserver.cc:460] Webserver started at http://127.24.0.189:38299/ using document root <none> and password file <none>
I20260812 01:19:26.040673 2791 fs_manager.cc:359] Metadata directory not provided
I20260812 01:19:26.040729 2791 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:19:26.041345 2791 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260812 01:19:26.041888 2991 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:26.042038 2791 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 01:19:26.042085 2791 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/master-1/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/master-1/wal
uuid: "35a50d99aa1b4d9c8a4b11849067ad0f"
format_stamp: "Formatted at 2026-08-12 01:19:25 on dist-test-slave-t3q3"
server_key: "239706af31e5d548ba29a94f363070f4"
server_key_iv: "0ac14dd942d48ae7dba426580778666e"
server_key_version: "encryptionkey@0"
I20260812 01:19:26.042141 2791 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:19:26.058070 2791 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:19:26.059808 2791 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:19:26.064248 2791 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.189:38795
I20260812 01:19:26.064409 3096 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.189:38795 every 8 connection(s)
I20260812 01:19:26.064643 2791 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/master-1/data/info.pb
I20260812 01:19:26.065739 3097 sys_catalog.cc:263] Verifying existing consensus state
I20260812 01:19:26.066323 3097 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 35a50d99aa1b4d9c8a4b11849067ad0f: Bootstrap starting.
I20260812 01:19:26.069607 3097 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 35a50d99aa1b4d9c8a4b11849067ad0f: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 01:19:26.069895 3097 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 35a50d99aa1b4d9c8a4b11849067ad0f: Bootstrap complete.
I20260812 01:19:26.070125 3097 raft_consensus.cc:348] T 00000000000000000000000000000000 P 35a50d99aa1b4d9c8a4b11849067ad0f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "efbfb4db2c524879a5d7d17801ff4cb0" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 46409 } } peers { permanent_uuid: "35a50d99aa1b4d9c8a4b11849067ad0f" member_type: NON_VOTER last_known_addr { host: "127.24.0.189" port: 38795 } attrs { promote: true } }
I20260812 01:19:26.070227 3097 raft_consensus.cc:729] T 00000000000000000000000000000000 P 35a50d99aa1b4d9c8a4b11849067ad0f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 35a50d99aa1b4d9c8a4b11849067ad0f, State: Initialized, Role: LEARNER
I20260812 01:19:26.070271 3097 consensus_queue.cc:260] T 00000000000000000000000000000000 P 35a50d99aa1b4d9c8a4b11849067ad0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "efbfb4db2c524879a5d7d17801ff4cb0" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 46409 } } peers { permanent_uuid: "35a50d99aa1b4d9c8a4b11849067ad0f" member_type: NON_VOTER last_known_addr { host: "127.24.0.189" port: 38795 } attrs { promote: true } }
I20260812 01:19:26.070494 3097 sys_catalog.cc:564] T 00000000000000000000000000000000 P 35a50d99aa1b4d9c8a4b11849067ad0f [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:19:26.070489 3100 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35a50d99aa1b4d9c8a4b11849067ad0f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "efbfb4db2c524879a5d7d17801ff4cb0" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 46409 } } peers { permanent_uuid: "35a50d99aa1b4d9c8a4b11849067ad0f" member_type: NON_VOTER last_known_addr { host: "127.24.0.189" port: 38795 } attrs { promote: true } } }
I20260812 01:19:26.070626 3100 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35a50d99aa1b4d9c8a4b11849067ad0f [sys.catalog]: This master's current role is: LEARNER
I20260812 01:19:26.072248 3118 catalog_manager.cc:1259] Loaded cluster ID: 702944a007944b5db36839527bbbf20f
I20260812 01:19:26.072299 3118 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 35a50d99aa1b4d9c8a4b11849067ad0f: loading cluster ID for follower catalog manager: success
I20260812 01:19:26.073292 3118 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 35a50d99aa1b4d9c8a4b11849067ad0f: acquiring CA information for follower catalog manager: success
I20260812 01:19:26.073573 3118 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 35a50d99aa1b4d9c8a4b11849067ad0f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 01:19:26.120048 3119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:26.112412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44116 (local address 127.24.0.189:38795)
0812 01:19:26.112558 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:26.112560 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:26.112573 (+ 13us) server_negotiation.cc:407] Connection header received
0812 01:19:26.112600 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:26.112601 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:26.112610 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:26.112695 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:26.113507 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.114290 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.115387 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.115590 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.115641 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:26.117317 (+ 1676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:26.117318 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:26.117318 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:26.117325 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:26.118825 (+ 1500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:26.119255 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:26.119256 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:26.119256 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:26.119311 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:26.119541 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:26.119542 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:26.119542 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:26.119641 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:26.119727 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:26.119886 (+ 159us) server_negotiation.cc:299] Negotiation successful
0812 01:19:26.119954 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":87,"threads_started":1}
WARNING: no policy specified for oryx/127.24.0.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.0.188@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.0.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/krb5kdc/oryx_127.24.0.188.keytab.
Entry for principal oryx/127.24.0.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/krb5kdc/oryx_127.24.0.188.keytab.
I20260812 01:19:26.147076 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.188:37741
--webserver_interface=127.24.0.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/krb5kdc/oryx_127.24.0.188.keytab
--principal=oryx/127.24.0.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:38987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.0.190:46409,127.24.0.189:38795,127.24.0.188:37741
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:19:26.231720 3123 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:19:26.231925 3123 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:19:26.231946 3123 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:19:26.233156 3123 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:19:26.233201 3123 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:19:26.233215 3123 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:19:26.233228 3123 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:19:26.233271 3123 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:19:26.234802 3123 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:38987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.24.0.190:46409,127.24.0.189:38795,127.24.0.188:37741
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/krb5kdc/oryx_127.24.0.188.keytab
--principal=oryx/127.24.0.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.188:37741
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.24.0.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3123
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:19:26.235066 3123 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:19:26.235313 3123 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:19:26.240435 3130 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:26.240471 3131 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:19:26.240643 3123 server_base.cc:1034] running on GCE node
W20260812 01:19:26.240731 3134 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:19:26.240978 3123 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:19:26.241298 3123 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:19:26.242447 3123 hybrid_clock.cc:648] HybridClock initialized: now 1786497566242425 us; error 40 us; skew 500 ppm
Aug 12 01:19:26 dist-test-slave-t3q3 krb5kdc[1963](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497566, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.0.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:19:26.245126 3123 init.cc:377] Logged in from keytab as oryx/127.24.0.188@KRBTEST.COM (short username oryx)
I20260812 01:19:26.245635 3123 webserver.cc:460] Webserver started at http://127.24.0.188:39355/ using document root <none> and password file <none>
I20260812 01:19:26.245877 3123 fs_manager.cc:359] Metadata directory not provided
I20260812 01:19:26.245946 3123 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:19:26.246101 3123 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:19:26.247255 3123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/master-2/data/instance:
uuid: "ccb9bfbd91334a59bb9b562bc5be7fa6"
format_stamp: "Formatted at 2026-08-12 01:19:26 on dist-test-slave-t3q3"
server_key: "bf445e23010670e7b588d7d113075dd4"
server_key_iv: "7c2903b727952cc70407c2adb6a0818f"
server_key_version: "encryptionkey@0"
I20260812 01:19:26.247641 3123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/master-2/wal/instance:
uuid: "ccb9bfbd91334a59bb9b562bc5be7fa6"
format_stamp: "Formatted at 2026-08-12 01:19:26 on dist-test-slave-t3q3"
server_key: "bf445e23010670e7b588d7d113075dd4"
server_key_iv: "7c2903b727952cc70407c2adb6a0818f"
server_key_version: "encryptionkey@0"
I20260812 01:19:26.249154 3123 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 01:19:26.249974 3143 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:26.250193 3123 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:19:26.250308 3123 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/master-2/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/master-2/wal
uuid: "ccb9bfbd91334a59bb9b562bc5be7fa6"
format_stamp: "Formatted at 2026-08-12 01:19:26 on dist-test-slave-t3q3"
server_key: "bf445e23010670e7b588d7d113075dd4"
server_key_iv: "7c2903b727952cc70407c2adb6a0818f"
server_key_version: "encryptionkey@0"
I20260812 01:19:26.250385 3123 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:19:26.270758 3123 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:19:26.273141 3123 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:19:26.273304 3123 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:19:26.277697 3123 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.188:37741
I20260812 01:19:26.277731 3225 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.188:37741 every 8 connection(s)
I20260812 01:19:26.278116 3123 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/master-2/data/info.pb
I20260812 01:19:26.279155 3226 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:19:26.280809 3226 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.0.190" port: 46409 } has no permanent_uuid. Determining permanent_uuid...
I20260812 01:19:26.282990 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 3123
Aug 12 01:19:26 dist-test-slave-t3q3 krb5kdc[1963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497566, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.0.188@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM
Aug 12 01:19:26 dist-test-slave-t3q3 krb5kdc[1963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497565, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.0.188@KRBTEST.COM
I20260812 01:19:26.291399 2108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:26.281603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60102 (local address 127.24.0.190:46409)
0812 01:19:26.281651 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:26.281653 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:26.281839 (+ 186us) server_negotiation.cc:407] Connection header received
0812 01:19:26.282339 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:26.282340 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:26.282351 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:26.282405 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:26.283022 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.283831 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.285117 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.285314 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.285369 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:26.288300 (+ 2931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:26.288301 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:26.288302 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:26.288309 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:26.289750 (+ 1441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:26.290279 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:26.290280 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:26.290280 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:26.290346 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:26.290665 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:26.290666 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:26.290666 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:26.290819 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:26.290900 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:26.291192 (+ 292us) server_negotiation.cc:299] Negotiation successful
0812 01:19:26.291277 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 01:19:26.291503 3228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:26.281485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:46409 (local address 127.0.0.1:60102)
0812 01:19:26.281715 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:26.281751 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:26.282243 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:26.282458 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:26.282462 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:26.282715 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:26.282929 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:26.282934 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.283909 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.283910 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:26.285027 (+ 1117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:26.285029 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.285407 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.285408 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:26.285540 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:26.286156 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:26.286165 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:26.288180 (+ 2015us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:19:26.289858 (+ 1678us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:26.289868 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:26.289870 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:26.290199 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:26.290447 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:26.290448 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:26.290448 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:26.290592 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:26.290904 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:19:26.290910 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:19:26.291046 (+ 136us) client_negotiation.cc:769] Sending connection context
0812 01:19:26.291138 (+ 92us) client_negotiation.cc:240] Negotiation successful
0812 01:19:26.291304 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":124,"threads_started":1}
I20260812 01:19:26.292019 3226 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.0.189" port: 38795 } has no permanent_uuid. Determining permanent_uuid...
I20260812 01:19:26.293004 3229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:26.283564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54942 (local address 127.24.0.188:37741)
0812 01:19:26.283775 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:26.283779 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:26.283794 (+ 15us) server_negotiation.cc:407] Connection header received
0812 01:19:26.283812 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:26.283814 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:26.283830 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:26.283930 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:26.284559 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.285534 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.286249 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.286682 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.286768 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:26.289597 (+ 2829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:26.289606 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:26.289607 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:26.289615 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:26.291565 (+ 1950us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:26.292026 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:26.292029 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:26.292029 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:26.292135 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:26.292400 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:26.292402 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:26.292402 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:26.292611 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:26.292759 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:26.292810 (+ 51us) server_negotiation.cc:299] Negotiation successful
0812 01:19:26.292892 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":100,"threads_started":1}
Aug 12 01:19:26 dist-test-slave-t3q3 krb5kdc[1963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497566, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.0.188@KRBTEST.COM for oryx/127.24.0.189@KRBTEST.COM
I20260812 01:19:26.298677 3228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:26.292306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.189:38795 (local address 127.0.0.1:44118)
0812 01:19:26.292357 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:26.292368 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:26.292407 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:26.292577 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:26.292578 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:26.292728 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:26.292784 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:26.292786 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.293902 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.293903 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:26.294401 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:26.294402 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.294754 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.294755 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:26.294806 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:26.295106 (+ 300us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:26.295110 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:26.296251 (+ 1141us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:19:26.297846 (+ 1595us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:26.297847 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:26.297847 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:26.298022 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:26.298218 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:26.298218 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:26.298219 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:26.298269 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:26.298463 (+ 194us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:19:26.298464 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:19:26.298502 (+ 38us) client_negotiation.cc:769] Sending connection context
0812 01:19:26.298553 (+ 51us) client_negotiation.cc:240] Negotiation successful
0812 01:19:26.298621 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260812 01:19:26.298698 3119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:26.292406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44118 (local address 127.24.0.189:38795)
0812 01:19:26.292451 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:26.292453 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:26.292461 (+ 8us) server_negotiation.cc:407] Connection header received
0812 01:19:26.292488 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:26.292488 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:26.292508 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:26.292550 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:26.292897 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.293539 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.294474 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.294685 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.294741 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:26.296332 (+ 1591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:26.296333 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:26.296333 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:26.296340 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:26.297769 (+ 1429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:26.298096 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:26.298097 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:26.298097 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:26.298148 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:26.298322 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:26.298322 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:26.298322 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:26.298415 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:26.298461 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:26.298568 (+ 107us) server_negotiation.cc:299] Negotiation successful
0812 01:19:26.298635 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 01:19:26.299276 3226 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.0.188" port: 37741 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 01:19:26 dist-test-slave-t3q3 krb5kdc[1963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497566, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.0.188@KRBTEST.COM for oryx/127.24.0.188@KRBTEST.COM
I20260812 01:19:26.305344 3228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:26.299551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.188:37741 (local address 127.0.0.1:54944)
0812 01:19:26.299600 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:26.299610 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:26.299672 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:26.299792 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:26.299793 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:26.299916 (+ 123us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:26.299941 (+ 25us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:26.299942 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.300610 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.300611 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:26.301061 (+ 450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:26.301063 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.301452 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.301453 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:26.301508 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:26.301811 (+ 303us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:26.301815 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:26.302882 (+ 1067us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:19:26.304513 (+ 1631us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:26.304514 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:26.304515 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:26.304687 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:26.304866 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:26.304866 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:26.304867 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:26.304919 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:26.305118 (+ 199us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:19:26.305118 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:19:26.305153 (+ 35us) client_negotiation.cc:769] Sending connection context
0812 01:19:26.305215 (+ 62us) client_negotiation.cc:240] Negotiation successful
0812 01:19:26.305288 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260812 01:19:26.305405 3229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:26.299618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54944 (local address 127.24.0.188:37741)
0812 01:19:26.299670 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:26.299672 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:26.299681 (+ 9us) server_negotiation.cc:407] Connection header received
0812 01:19:26.299723 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:26.299724 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:26.299733 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:26.299788 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:26.300040 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.300521 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.301156 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.301368 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.301452 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:26.302959 (+ 1507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:26.302960 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:26.302960 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:26.302966 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:26.304400 (+ 1434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:26.304752 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:26.304752 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:26.304753 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:26.304803 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:26.304972 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:26.304972 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:26.304972 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:26.305063 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:26.305112 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:26.305230 (+ 118us) server_negotiation.cc:299] Negotiation successful
0812 01:19:26.305332 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":46,"server-negotiator.queue_time_us":35}
I20260812 01:19:26.306800 3226 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ccb9bfbd91334a59bb9b562bc5be7fa6: Bootstrap starting.
I20260812 01:19:26.307610 3226 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ccb9bfbd91334a59bb9b562bc5be7fa6: Neither blocks nor log segments found. Creating new log.
I20260812 01:19:26.307873 3226 log.cc:824] T 00000000000000000000000000000000 P ccb9bfbd91334a59bb9b562bc5be7fa6: Log is configured to *not* fsync() on all Append() calls
I20260812 01:19:26.308657 3226 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ccb9bfbd91334a59bb9b562bc5be7fa6: No bootstrap required, opened a new log
I20260812 01:19:26.310035 3226 raft_consensus.cc:348] T 00000000000000000000000000000000 P ccb9bfbd91334a59bb9b562bc5be7fa6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efbfb4db2c524879a5d7d17801ff4cb0" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 46409 } } peers { permanent_uuid: "35a50d99aa1b4d9c8a4b11849067ad0f" member_type: VOTER last_known_addr { host: "127.24.0.189" port: 38795 } } peers { permanent_uuid: "ccb9bfbd91334a59bb9b562bc5be7fa6" member_type: VOTER last_known_addr { host: "127.24.0.188" port: 37741 } }
I20260812 01:19:26.310156 3226 raft_consensus.cc:374] T 00000000000000000000000000000000 P ccb9bfbd91334a59bb9b562bc5be7fa6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:19:26.310187 3226 raft_consensus.cc:729] T 00000000000000000000000000000000 P ccb9bfbd91334a59bb9b562bc5be7fa6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccb9bfbd91334a59bb9b562bc5be7fa6, State: Initialized, Role: FOLLOWER
I20260812 01:19:26.310317 3226 consensus_queue.cc:260] T 00000000000000000000000000000000 P ccb9bfbd91334a59bb9b562bc5be7fa6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efbfb4db2c524879a5d7d17801ff4cb0" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 46409 } } peers { permanent_uuid: "35a50d99aa1b4d9c8a4b11849067ad0f" member_type: VOTER last_known_addr { host: "127.24.0.189" port: 38795 } } peers { permanent_uuid: "ccb9bfbd91334a59bb9b562bc5be7fa6" member_type: VOTER last_known_addr { host: "127.24.0.188" port: 37741 } }
I20260812 01:19:26.310722 3226 sys_catalog.cc:564] T 00000000000000000000000000000000 P ccb9bfbd91334a59bb9b562bc5be7fa6 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:19:26.310704 3232 sys_catalog.cc:455] T 00000000000000000000000000000000 P ccb9bfbd91334a59bb9b562bc5be7fa6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efbfb4db2c524879a5d7d17801ff4cb0" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 46409 } } peers { permanent_uuid: "35a50d99aa1b4d9c8a4b11849067ad0f" member_type: VOTER last_known_addr { host: "127.24.0.189" port: 38795 } } peers { permanent_uuid: "ccb9bfbd91334a59bb9b562bc5be7fa6" member_type: VOTER last_known_addr { host: "127.24.0.188" port: 37741 } } }
I20260812 01:19:26.310868 3232 sys_catalog.cc:458] T 00000000000000000000000000000000 P ccb9bfbd91334a59bb9b562bc5be7fa6 [sys.catalog]: This master's current role is: FOLLOWER
W20260812 01:19:26.312659 3248 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ccb9bfbd91334a59bb9b562bc5be7fa6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 01:19:26.312724 3248 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 01:19:26.315126 3123 master_runner.cc:428] Detected that this master ccb9bfbd91334a59bb9b562bc5be7fa6 is joining an existing cluster
I20260812 01:19:26.315192 3123 master_runner.cc:432] Initiating AddMaster RPC to add 127.24.0.188:37741
I20260812 01:19:26.323534 2108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:26.316520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60108 (local address 127.24.0.190:46409)
0812 01:19:26.316574 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:26.316576 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:26.316709 (+ 133us) server_negotiation.cc:407] Connection header received
0812 01:19:26.316774 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:26.316776 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:26.316788 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:26.316851 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:26.317382 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.318148 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.319359 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.319652 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.319695 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:26.320876 (+ 1181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:26.320878 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:26.320878 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:26.320885 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:26.322338 (+ 1453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:26.322735 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:26.322736 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:26.322736 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:26.322788 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:26.323026 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:26.323027 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:26.323028 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:26.323168 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:26.323237 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:26.323406 (+ 169us) server_negotiation.cc:299] Negotiation successful
0812 01:19:26.323471 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 01:19:26.323909 3259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:26.316453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:46409 (local address 127.0.0.1:60108)
0812 01:19:26.316630 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:26.316642 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:26.316705 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:26.316904 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:26.316905 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:26.317193 (+ 288us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:26.317278 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:26.317280 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.318251 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.318251 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:26.319252 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:26.319256 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.319778 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.319779 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:26.319843 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:26.320324 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:26.320329 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:26.320772 (+ 443us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:19:26.322425 (+ 1653us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:26.322426 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:26.322426 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:26.322656 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:26.322854 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:26.322855 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:26.322855 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:26.322926 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:26.323249 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:19:26.323250 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:19:26.323307 (+ 57us) client_negotiation.cc:769] Sending connection context
0812 01:19:26.323391 (+ 84us) client_negotiation.cc:240] Negotiation successful
0812 01:19:26.323497 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"mutex_wait_us":38,"thread_start_us":109,"threads_started":1}
Aug 12 01:19:26 dist-test-slave-t3q3 krb5kdc[1963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497565, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.0.190@KRBTEST.COM for oryx/127.24.0.188@KRBTEST.COM
I20260812 01:19:26.335476 2947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:26.326035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.188:37741 (local address 127.0.0.1:54960)
0812 01:19:26.326085 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:26.326094 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:26.326134 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:26.326377 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:26.326378 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:26.326530 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:26.326565 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:26.326567 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.327229 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.327230 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:26.327734 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:26.327736 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.328189 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.328190 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:26.328259 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:26.328651 (+ 392us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:26.328656 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:26.330085 (+ 1429us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:19:26.334210 (+ 4125us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:26.334212 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:26.334212 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:26.334462 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:26.334741 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:26.334742 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:26.334742 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:26.334811 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:26.335128 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:19:26.335130 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:19:26.335180 (+ 50us) client_negotiation.cc:769] Sending connection context
0812 01:19:26.335235 (+ 55us) client_negotiation.cc:240] Negotiation successful
0812 01:19:26.335348 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260812 01:19:26.335767 3229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:26.317222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54950 (local address 127.24.0.188:37741)
0812 01:19:26.317258 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:26.317260 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:26.317593 (+ 333us) server_negotiation.cc:407] Connection header received
0812 01:19:26.317668 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:26.317669 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:26.317676 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:26.317701 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:26.318060 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.318732 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.319630 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.319886 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.319924 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:26.333135 (+ 13211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:26.333136 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:26.333136 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:26.333142 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:26.334691 (+ 1549us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:26.335076 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:26.335077 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:26.335077 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:26.335123 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:26.335293 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:26.335294 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:26.335294 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:26.335398 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:26.335435 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:26.335600 (+ 165us) server_negotiation.cc:299] Negotiation successful
0812 01:19:26.335676 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260812 01:19:26.336114 3262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:26.317288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.188:37741 (local address 127.0.0.1:54950)
0812 01:19:26.317544 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:26.317553 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:26.317612 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:26.317746 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:26.317747 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:26.317930 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:26.317988 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:26.317990 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.318813 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.318814 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:26.319554 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:26.319555 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.319985 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.319986 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:26.320047 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:26.332514 (+ 12467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:26.332519 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:26.333028 (+ 509us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:19:26.334779 (+ 1751us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:26.334779 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:26.334780 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:26.334987 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:26.335185 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:26.335186 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:26.335186 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:26.335233 (+ 47us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:26.335472 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:19:26.335473 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:19:26.335504 (+ 31us) client_negotiation.cc:769] Sending connection context
0812 01:19:26.335837 (+ 333us) client_negotiation.cc:240] Negotiation successful
0812 01:19:26.335950 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"mutex_wait_us":23,"thread_start_us":137,"threads_started":1}
I20260812 01:19:26.336050 3264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:26.326106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54960 (local address 127.24.0.188:37741)
0812 01:19:26.326252 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:26.326254 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:26.326259 (+ 5us) server_negotiation.cc:407] Connection header received
0812 01:19:26.326288 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:26.326289 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:26.326297 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:26.326373 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:26.326660 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.327135 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.327821 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.328034 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.328190 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:26.330216 (+ 2026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:26.330218 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:26.330218 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:26.330224 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:26.334116 (+ 3892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:26.334553 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:26.334554 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:26.334554 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:26.334626 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:26.334895 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:26.334896 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:26.334896 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:26.335042 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:26.335757 (+ 715us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:26.335800 (+ 43us) server_negotiation.cc:299] Negotiation successful
0812 01:19:26.335893 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":87,"threads_started":1}
I20260812 01:19:26.336997 2035 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.24.0.188:37741: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "ccb9bfbd91334a59bb9b562bc5be7fa6"
member_type: NON_VOTER
last_known_addr {
host: "127.24.0.188"
port: 37741
}
attrs {
promote: true
}
}
dest_uuid: "efbfb4db2c524879a5d7d17801ff4cb0"
cas_config_opid_index: 5
I20260812 01:19:26.337261 2035 consensus_queue.cc:237] T 00000000000000000000000000000000 P efbfb4db2c524879a5d7d17801ff4cb0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "efbfb4db2c524879a5d7d17801ff4cb0" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 46409 } } peers { permanent_uuid: "35a50d99aa1b4d9c8a4b11849067ad0f" member_type: NON_VOTER last_known_addr { host: "127.24.0.189" port: 38795 } attrs { promote: true } } peers { permanent_uuid: "ccb9bfbd91334a59bb9b562bc5be7fa6" member_type: NON_VOTER last_known_addr { host: "127.24.0.188" port: 37741 } attrs { promote: true } }
I20260812 01:19:26.337692 2951 raft_consensus.cc:2932] T 00000000000000000000000000000000 P efbfb4db2c524879a5d7d17801ff4cb0 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER ccb9bfbd91334a59bb9b562bc5be7fa6 (127.24.0.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "efbfb4db2c524879a5d7d17801ff4cb0" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 46409 } } peers { permanent_uuid: "35a50d99aa1b4d9c8a4b11849067ad0f" member_type: NON_VOTER last_known_addr { host: "127.24.0.189" port: 38795 } attrs { promote: true } } peers { permanent_uuid: "ccb9bfbd91334a59bb9b562bc5be7fa6" member_type: NON_VOTER last_known_addr { host: "127.24.0.188" port: 37741 } attrs { promote: true } } }
I20260812 01:19:26.338344 3186 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ccb9bfbd91334a59bb9b562bc5be7fa6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:19:26.338788 2951 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.24.0.188:37741
I20260812 01:19:26.338858 2951 sys_catalog.cc:455] T 00000000000000000000000000000000 P efbfb4db2c524879a5d7d17801ff4cb0 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "efbfb4db2c524879a5d7d17801ff4cb0" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "efbfb4db2c524879a5d7d17801ff4cb0" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 46409 } } peers { permanent_uuid: "35a50d99aa1b4d9c8a4b11849067ad0f" member_type: NON_VOTER last_known_addr { host: "127.24.0.189" port: 38795 } attrs { promote: true } } peers { permanent_uuid: "ccb9bfbd91334a59bb9b562bc5be7fa6" member_type: NON_VOTER last_known_addr { host: "127.24.0.188" port: 37741 } attrs { promote: true } } }
I20260812 01:19:26.338918 2951 sys_catalog.cc:458] T 00000000000000000000000000000000 P efbfb4db2c524879a5d7d17801ff4cb0 [sys.catalog]: This master's current role is: LEADER
I20260812 01:19:26.339143 3123 master.cc:537] Master@127.24.0.188:37741 shutting down...
I20260812 01:19:26.339279 3186 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ccb9bfbd91334a59bb9b562bc5be7fa6 [term 1 FOLLOWER]: Refusing update from remote peer efbfb4db2c524879a5d7d17801ff4cb0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260812 01:19:26.339567 2950 consensus_queue.cc:1035] T 00000000000000000000000000000000 P efbfb4db2c524879a5d7d17801ff4cb0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccb9bfbd91334a59bb9b562bc5be7fa6" member_type: NON_VOTER last_known_addr { host: "127.24.0.188" port: 37741 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 01:19:26.340586 2010 consensus_peers.cc:487] T 00000000000000000000000000000000 P efbfb4db2c524879a5d7d17801ff4cb0 -> Peer ccb9bfbd91334a59bb9b562bc5be7fa6 (127.24.0.188:37741): Couldn't send request to peer ccb9bfbd91334a59bb9b562bc5be7fa6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260812 01:19:26.341017 3261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:26.316736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.189:38795 (local address 127.0.0.1:44120)
0812 01:19:26.316843 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:26.316852 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:26.316894 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:26.317056 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:26.317057 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:26.317325 (+ 268us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:26.317420 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:26.317424 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.318372 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.318373 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:26.319237 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:26.319239 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.319757 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.319758 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:26.319813 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:26.332823 (+ 13010us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:26.332830 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:26.333804 (+ 974us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:19:26.336718 (+ 2914us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:26.336718 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:26.336719 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:26.336942 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:26.337194 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:26.337195 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:26.337195 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:26.337272 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:26.340614 (+ 3342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:19:26.340615 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:19:26.340665 (+ 50us) client_negotiation.cc:769] Sending connection context
0812 01:19:26.340749 (+ 84us) client_negotiation.cc:240] Negotiation successful
0812 01:19:26.340872 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":70,"threads_started":1}
I20260812 01:19:26.341681 3119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:26.316902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44120 (local address 127.24.0.189:38795)
0812 01:19:26.316942 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:26.316944 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:26.316951 (+ 7us) server_negotiation.cc:407] Connection header received
0812 01:19:26.316971 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:26.316972 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:26.316980 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:26.317052 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:26.317493 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.318284 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.319322 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.319628 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.319694 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:26.334202 (+ 14508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:26.334203 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:26.334203 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:26.334212 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:26.336001 (+ 1789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:26.337028 (+ 1027us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:26.337029 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:26.337029 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:26.337091 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:26.337428 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:26.337430 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:26.337430 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:26.337585 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:26.337656 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:26.341459 (+ 3803us) server_negotiation.cc:299] Negotiation successful
0812 01:19:26.341560 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 01:19:26.342327 2947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:26.337704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.189:38795 (local address 127.0.0.1:44126)
0812 01:19:26.337918 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:26.337930 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:26.337972 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:26.338222 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:26.338223 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:26.338225 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:19:26.338260 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:26.338261 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.339215 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.339216 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:26.341021 (+ 1805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:26.341024 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.342014 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.342016 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:26.342081 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:26.342108 (+ 27us) client_negotiation.cc:769] Sending connection context
0812 01:19:26.342200 (+ 92us) client_negotiation.cc:240] Negotiation successful
0812 01:19:26.342242 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198}
I20260812 01:19:26.342350 3267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:26.337797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44126 (local address 127.24.0.189:38795)
0812 01:19:26.338058 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:26.338060 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:26.338067 (+ 7us) server_negotiation.cc:407] Connection header received
0812 01:19:26.338088 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:26.338089 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:26.338115 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:26.338317 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:19:26.338350 (+ 33us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.339115 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.341172 (+ 2057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.341919 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.342003 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:26.342099 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:26.342237 (+ 138us) server_negotiation.cc:299] Negotiation successful
0812 01:19:26.342281 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":243,"thread_start_us":76,"threads_started":1}
I20260812 01:19:26.342712 3049 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 35a50d99aa1b4d9c8a4b11849067ad0f [term 1 LEARNER]: Refusing update from remote peer efbfb4db2c524879a5d7d17801ff4cb0: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260812 01:19:26.342900 2950 consensus_queue.cc:1035] T 00000000000000000000000000000000 P efbfb4db2c524879a5d7d17801ff4cb0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35a50d99aa1b4d9c8a4b11849067ad0f" member_type: NON_VOTER last_known_addr { host: "127.24.0.189" port: 38795 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260812 01:19:26.343464 3049 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 35a50d99aa1b4d9c8a4b11849067ad0f [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER ccb9bfbd91334a59bb9b562bc5be7fa6 (127.24.0.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "efbfb4db2c524879a5d7d17801ff4cb0" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 46409 } } peers { permanent_uuid: "35a50d99aa1b4d9c8a4b11849067ad0f" member_type: NON_VOTER last_known_addr { host: "127.24.0.189" port: 38795 } attrs { promote: true } } peers { permanent_uuid: "ccb9bfbd91334a59bb9b562bc5be7fa6" member_type: NON_VOTER last_known_addr { host: "127.24.0.188" port: 37741 } attrs { promote: true } } }
I20260812 01:19:26.344238 3123 raft_consensus.cc:2229] T 00000000000000000000000000000000 P ccb9bfbd91334a59bb9b562bc5be7fa6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 01:19:26.344333 3123 raft_consensus.cc:2258] T 00000000000000000000000000000000 P ccb9bfbd91334a59bb9b562bc5be7fa6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:19:26.344385 3123 tablet_replica.cc:330] T 00000000000000000000000000000000 P ccb9bfbd91334a59bb9b562bc5be7fa6: stopping tablet replica
I20260812 01:19:26.344907 3100 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35a50d99aa1b4d9c8a4b11849067ad0f [sys.catalog]: SysCatalogTable state changed. Reason: New leader efbfb4db2c524879a5d7d17801ff4cb0. Latest consensus state: current_term: 1 leader_uuid: "efbfb4db2c524879a5d7d17801ff4cb0" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "efbfb4db2c524879a5d7d17801ff4cb0" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 46409 } } peers { permanent_uuid: "35a50d99aa1b4d9c8a4b11849067ad0f" member_type: NON_VOTER last_known_addr { host: "127.24.0.189" port: 38795 } attrs { promote: true } } peers { permanent_uuid: "ccb9bfbd91334a59bb9b562bc5be7fa6" member_type: NON_VOTER last_known_addr { host: "127.24.0.188" port: 37741 } attrs { promote: true } } }
I20260812 01:19:26.345006 3100 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35a50d99aa1b4d9c8a4b11849067ad0f [sys.catalog]: This master's current role is: LEARNER
I20260812 01:19:26.345055 3100 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35a50d99aa1b4d9c8a4b11849067ad0f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "efbfb4db2c524879a5d7d17801ff4cb0" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "efbfb4db2c524879a5d7d17801ff4cb0" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 46409 } } peers { permanent_uuid: "35a50d99aa1b4d9c8a4b11849067ad0f" member_type: NON_VOTER last_known_addr { host: "127.24.0.189" port: 38795 } attrs { promote: true } } peers { permanent_uuid: "ccb9bfbd91334a59bb9b562bc5be7fa6" member_type: NON_VOTER last_known_addr { host: "127.24.0.188" port: 37741 } attrs { promote: true } } }
I20260812 01:19:26.345098 3100 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35a50d99aa1b4d9c8a4b11849067ad0f [sys.catalog]: This master's current role is: LEARNER
I20260812 01:19:26.345311 2950 raft_consensus.cc:1050] T 00000000000000000000000000000000 P efbfb4db2c524879a5d7d17801ff4cb0: attempting to promote NON_VOTER 35a50d99aa1b4d9c8a4b11849067ad0f to VOTER
I20260812 01:19:26.345503 2950 consensus_queue.cc:237] T 00000000000000000000000000000000 P efbfb4db2c524879a5d7d17801ff4cb0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "efbfb4db2c524879a5d7d17801ff4cb0" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 46409 } } peers { permanent_uuid: "35a50d99aa1b4d9c8a4b11849067ad0f" member_type: VOTER last_known_addr { host: "127.24.0.189" port: 38795 } attrs { promote: false } } peers { permanent_uuid: "ccb9bfbd91334a59bb9b562bc5be7fa6" member_type: NON_VOTER last_known_addr { host: "127.24.0.188" port: 37741 } attrs { promote: true } }
W20260812 01:19:26.346279 3147 connection.cc:441] server connection from 127.0.0.1:54942 torn down before Call kudu.master.MasterService.ListTables from 127.0.0.1:54942 (request call id 9) could send its response
I20260812 01:19:26.346367 3049 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 35a50d99aa1b4d9c8a4b11849067ad0f [term 1 LEARNER]: Refusing update from remote peer efbfb4db2c524879a5d7d17801ff4cb0: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260812 01:19:26.346551 2951 consensus_queue.cc:1035] T 00000000000000000000000000000000 P efbfb4db2c524879a5d7d17801ff4cb0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35a50d99aa1b4d9c8a4b11849067ad0f" member_type: VOTER last_known_addr { host: "127.24.0.189" port: 38795 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260812 01:19:26.346807 2010 connection.cc:535] client connection to 127.24.0.188:37741 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260812 01:19:26.346858 2010 consensus_peers.cc:487] T 00000000000000000000000000000000 P efbfb4db2c524879a5d7d17801ff4cb0 -> Peer ccb9bfbd91334a59bb9b562bc5be7fa6 (127.24.0.188:37741): Couldn't send request to peer ccb9bfbd91334a59bb9b562bc5be7fa6. Status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107). This is attempt 1: this message will repeat every 5th retry.
I20260812 01:19:26.347426 3272 raft_consensus.cc:2932] T 00000000000000000000000000000000 P efbfb4db2c524879a5d7d17801ff4cb0 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 35a50d99aa1b4d9c8a4b11849067ad0f (127.24.0.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "efbfb4db2c524879a5d7d17801ff4cb0" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 46409 } } peers { permanent_uuid: "35a50d99aa1b4d9c8a4b11849067ad0f" member_type: VOTER last_known_addr { host: "127.24.0.189" port: 38795 } attrs { promote: false } } peers { permanent_uuid: "ccb9bfbd91334a59bb9b562bc5be7fa6" member_type: NON_VOTER last_known_addr { host: "127.24.0.188" port: 37741 } attrs { promote: true } } }
I20260812 01:19:26.347571 3049 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 35a50d99aa1b4d9c8a4b11849067ad0f [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 35a50d99aa1b4d9c8a4b11849067ad0f (127.24.0.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "efbfb4db2c524879a5d7d17801ff4cb0" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 46409 } } peers { permanent_uuid: "35a50d99aa1b4d9c8a4b11849067ad0f" member_type: VOTER last_known_addr { host: "127.24.0.189" port: 38795 } attrs { promote: false } } peers { permanent_uuid: "ccb9bfbd91334a59bb9b562bc5be7fa6" member_type: NON_VOTER last_known_addr { host: "127.24.0.188" port: 37741 } attrs { promote: true } } }
I20260812 01:19:26.348343 3272 sys_catalog.cc:455] T 00000000000000000000000000000000 P efbfb4db2c524879a5d7d17801ff4cb0 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "efbfb4db2c524879a5d7d17801ff4cb0" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "efbfb4db2c524879a5d7d17801ff4cb0" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 46409 } } peers { permanent_uuid: "35a50d99aa1b4d9c8a4b11849067ad0f" member_type: VOTER last_known_addr { host: "127.24.0.189" port: 38795 } attrs { promote: false } } peers { permanent_uuid: "ccb9bfbd91334a59bb9b562bc5be7fa6" member_type: NON_VOTER last_known_addr { host: "127.24.0.188" port: 37741 } attrs { promote: true } } }
I20260812 01:19:26.348394 2951 sys_catalog.cc:455] T 00000000000000000000000000000000 P efbfb4db2c524879a5d7d17801ff4cb0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "efbfb4db2c524879a5d7d17801ff4cb0" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "efbfb4db2c524879a5d7d17801ff4cb0" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 46409 } } peers { permanent_uuid: "35a50d99aa1b4d9c8a4b11849067ad0f" member_type: VOTER last_known_addr { host: "127.24.0.189" port: 38795 } attrs { promote: false } } peers { permanent_uuid: "ccb9bfbd91334a59bb9b562bc5be7fa6" member_type: NON_VOTER last_known_addr { host: "127.24.0.188" port: 37741 } attrs { promote: true } } }
I20260812 01:19:26.348469 2951 sys_catalog.cc:458] T 00000000000000000000000000000000 P efbfb4db2c524879a5d7d17801ff4cb0 [sys.catalog]: This master's current role is: LEADER
I20260812 01:19:26.348433 3272 sys_catalog.cc:458] T 00000000000000000000000000000000 P efbfb4db2c524879a5d7d17801ff4cb0 [sys.catalog]: This master's current role is: LEADER
I20260812 01:19:26.348668 3270 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35a50d99aa1b4d9c8a4b11849067ad0f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "efbfb4db2c524879a5d7d17801ff4cb0" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "efbfb4db2c524879a5d7d17801ff4cb0" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 46409 } } peers { permanent_uuid: "35a50d99aa1b4d9c8a4b11849067ad0f" member_type: VOTER last_known_addr { host: "127.24.0.189" port: 38795 } attrs { promote: false } } peers { permanent_uuid: "ccb9bfbd91334a59bb9b562bc5be7fa6" member_type: NON_VOTER last_known_addr { host: "127.24.0.188" port: 37741 } attrs { promote: true } } }
I20260812 01:19:26.348760 3270 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35a50d99aa1b4d9c8a4b11849067ad0f [sys.catalog]: This master's current role is: FOLLOWER
I20260812 01:19:26.358173 3123 master.cc:552] Master@127.24.0.188:37741 shutdown complete.
I20260812 01:19:26.358234 3123 master_runner.cc:305] Clearing existing system tablet
I20260812 01:19:26.358371 3123 fs_manager.cc:359] Metadata directory not provided
I20260812 01:19:26.358420 3123 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:19:26.359196 3123 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:19:26.359762 3277 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:26.359927 3123 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:19:26.360006 3123 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 01:19:26.360033 3123 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/master-2/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/master-2/wal
uuid: "ccb9bfbd91334a59bb9b562bc5be7fa6"
format_stamp: "Formatted at 2026-08-12 01:19:26 on dist-test-slave-t3q3"
server_key: "bf445e23010670e7b588d7d113075dd4"
server_key_iv: "7c2903b727952cc70407c2adb6a0818f"
server_key_version: "encryptionkey@0"
I20260812 01:19:26.361056 3123 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P ccb9bfbd91334a59bb9b562bc5be7fa6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:19:26.362583 3123 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P ccb9bfbd91334a59bb9b562bc5be7fa6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260812 01:19:26.362661 3123 log.cc:1196] T 00000000000000000000000000000000 P ccb9bfbd91334a59bb9b562bc5be7fa6: Deleting WAL directory at /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260812 01:19:26.362879 3123 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P ccb9bfbd91334a59bb9b562bc5be7fa6: Deleting consensus metadata
I20260812 01:19:26.363152 3123 master_runner.cc:315] Copying system tablet from 127.24.0.190:46409
I20260812 01:19:26.363799 3123 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P ccb9bfbd91334a59bb9b562bc5be7fa6: tablet copy: Beginning tablet copy session from remote peer at address 127.24.0.190:46409
I20260812 01:19:26.370501 2108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:26.364211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60110 (local address 127.24.0.190:46409)
0812 01:19:26.364285 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:26.364287 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:26.364432 (+ 145us) server_negotiation.cc:407] Connection header received
0812 01:19:26.364503 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:26.364503 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:26.364513 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:26.364560 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:26.364940 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.365472 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.366530 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.366732 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.366784 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:26.367852 (+ 1068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:26.367853 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:26.367853 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:26.367860 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:26.369400 (+ 1540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:26.369796 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:26.369797 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:26.369797 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:26.369847 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:26.370045 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:26.370045 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:26.370045 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:26.370140 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:26.370202 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:26.370371 (+ 169us) server_negotiation.cc:299] Negotiation successful
0812 01:19:26.370438 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 01:19:26.370575 3285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:26.364139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:46409 (local address 127.0.0.1:60110)
0812 01:19:26.364342 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:26.364354 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:26.364436 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:26.364578 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:26.364579 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:26.364800 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:26.364863 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:26.364865 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.365555 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.365556 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:26.366440 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:26.366443 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.366810 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.366812 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:26.366885 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:26.367354 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:26.367359 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:26.367760 (+ 401us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:19:26.369485 (+ 1725us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:26.369487 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:26.369487 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:26.369714 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:26.369910 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:26.369911 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:26.369912 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:26.369981 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:26.370216 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:19:26.370217 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:19:26.370265 (+ 48us) client_negotiation.cc:769] Sending connection context
0812 01:19:26.370341 (+ 76us) client_negotiation.cc:240] Negotiation successful
0812 01:19:26.370448 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":96,"threads_started":1}
I20260812 01:19:26.370836 2066 tablet_copy_service.cc:140] P efbfb4db2c524879a5d7d17801ff4cb0: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer ccb9bfbd91334a59bb9b562bc5be7fa6 ({username='oryx', principal='oryx/127.24.0.188@KRBTEST.COM'} at 127.0.0.1:60110)
I20260812 01:19:26.370891 2066 tablet_copy_service.cc:161] P efbfb4db2c524879a5d7d17801ff4cb0: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer ccb9bfbd91334a59bb9b562bc5be7fa6 at {username='oryx', principal='oryx/127.24.0.188@KRBTEST.COM'} at 127.0.0.1:60110: session id = ccb9bfbd91334a59bb9b562bc5be7fa6-00000000000000000000000000000000
I20260812 01:19:26.371405 2066 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P efbfb4db2c524879a5d7d17801ff4cb0: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 01:19:26.372004 3123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260812 01:19:26.373042 3123 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P ccb9bfbd91334a59bb9b562bc5be7fa6: tablet copy: Starting download of 0 data blocks...
I20260812 01:19:26.373203 3123 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P ccb9bfbd91334a59bb9b562bc5be7fa6: tablet copy: Starting download of 1 WAL segments...
I20260812 01:19:26.374027 3123 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P ccb9bfbd91334a59bb9b562bc5be7fa6: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 01:19:26.374738 2066 tablet_copy_service.cc:342] P efbfb4db2c524879a5d7d17801ff4cb0: Request end of tablet copy session ccb9bfbd91334a59bb9b562bc5be7fa6-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.24.0.188@KRBTEST.COM'} at 127.0.0.1:60110
I20260812 01:19:26.374797 2066 tablet_copy_service.cc:433] P efbfb4db2c524879a5d7d17801ff4cb0: ending tablet copy session ccb9bfbd91334a59bb9b562bc5be7fa6-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer ccb9bfbd91334a59bb9b562bc5be7fa6
I20260812 01:19:26.377280 3123 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:19:26.378427 3287 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:26.378505 3288 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:26.381023 3292 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:19:26.381153 3123 server_base.cc:1034] running on GCE node
I20260812 01:19:26.381289 3123 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:19:26.381438 3123 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:19:26.382546 3123 hybrid_clock.cc:648] HybridClock initialized: now 1786497566382537 us; error 25 us; skew 500 ppm
Aug 12 01:19:26 dist-test-slave-t3q3 krb5kdc[1963](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497566, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.0.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:19:26.383991 3123 init.cc:377] Logged in from keytab as oryx/127.24.0.188@KRBTEST.COM (short username oryx)
I20260812 01:19:26.384276 3123 webserver.cc:460] Webserver started at http://127.24.0.188:42933/ using document root <none> and password file <none>
I20260812 01:19:26.384393 3123 fs_manager.cc:359] Metadata directory not provided
I20260812 01:19:26.384426 3123 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:19:26.385107 3123 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260812 01:19:26.385674 3302 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:26.385821 3123 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 01:19:26.385877 3123 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/master-2/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/master-2/wal
uuid: "ccb9bfbd91334a59bb9b562bc5be7fa6"
format_stamp: "Formatted at 2026-08-12 01:19:26 on dist-test-slave-t3q3"
server_key: "bf445e23010670e7b588d7d113075dd4"
server_key_iv: "7c2903b727952cc70407c2adb6a0818f"
server_key_version: "encryptionkey@0"
I20260812 01:19:26.385927 3123 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:19:26.394156 3123 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:19:26.395721 3123 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:19:26.399287 3123 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.188:37741
I20260812 01:19:26.399595 3380 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.188:37741 every 8 connection(s)
I20260812 01:19:26.399696 3123 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/master-2/data/info.pb
I20260812 01:19:26.400471 3381 sys_catalog.cc:263] Verifying existing consensus state
I20260812 01:19:26.400861 3381 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ccb9bfbd91334a59bb9b562bc5be7fa6: Bootstrap starting.
I20260812 01:19:26.403224 3381 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ccb9bfbd91334a59bb9b562bc5be7fa6: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 01:19:26.403476 3381 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ccb9bfbd91334a59bb9b562bc5be7fa6: Bootstrap complete.
I20260812 01:19:26.403715 3381 raft_consensus.cc:348] T 00000000000000000000000000000000 P ccb9bfbd91334a59bb9b562bc5be7fa6 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "efbfb4db2c524879a5d7d17801ff4cb0" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 46409 } } peers { permanent_uuid: "35a50d99aa1b4d9c8a4b11849067ad0f" member_type: VOTER last_known_addr { host: "127.24.0.189" port: 38795 } attrs { promote: false } } peers { permanent_uuid: "ccb9bfbd91334a59bb9b562bc5be7fa6" member_type: NON_VOTER last_known_addr { host: "127.24.0.188" port: 37741 } attrs { promote: true } }
I20260812 01:19:26.403820 3381 raft_consensus.cc:729] T 00000000000000000000000000000000 P ccb9bfbd91334a59bb9b562bc5be7fa6 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ccb9bfbd91334a59bb9b562bc5be7fa6, State: Initialized, Role: LEARNER
I20260812 01:19:26.403872 3381 consensus_queue.cc:260] T 00000000000000000000000000000000 P ccb9bfbd91334a59bb9b562bc5be7fa6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "efbfb4db2c524879a5d7d17801ff4cb0" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 46409 } } peers { permanent_uuid: "35a50d99aa1b4d9c8a4b11849067ad0f" member_type: VOTER last_known_addr { host: "127.24.0.189" port: 38795 } attrs { promote: false } } peers { permanent_uuid: "ccb9bfbd91334a59bb9b562bc5be7fa6" member_type: NON_VOTER last_known_addr { host: "127.24.0.188" port: 37741 } attrs { promote: true } }
I20260812 01:19:26.404038 3381 sys_catalog.cc:564] T 00000000000000000000000000000000 P ccb9bfbd91334a59bb9b562bc5be7fa6 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:19:26.404050 3388 sys_catalog.cc:455] T 00000000000000000000000000000000 P ccb9bfbd91334a59bb9b562bc5be7fa6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "efbfb4db2c524879a5d7d17801ff4cb0" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 46409 } } peers { permanent_uuid: "35a50d99aa1b4d9c8a4b11849067ad0f" member_type: VOTER last_known_addr { host: "127.24.0.189" port: 38795 } attrs { promote: false } } peers { permanent_uuid: "ccb9bfbd91334a59bb9b562bc5be7fa6" member_type: NON_VOTER last_known_addr { host: "127.24.0.188" port: 37741 } attrs { promote: true } } }
I20260812 01:19:26.404181 3388 sys_catalog.cc:458] T 00000000000000000000000000000000 P ccb9bfbd91334a59bb9b562bc5be7fa6 [sys.catalog]: This master's current role is: LEARNER
I20260812 01:19:26.406214 3405 catalog_manager.cc:1259] Loaded cluster ID: 702944a007944b5db36839527bbbf20f
I20260812 01:19:26.406267 3405 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ccb9bfbd91334a59bb9b562bc5be7fa6: loading cluster ID for follower catalog manager: success
I20260812 01:19:26.407325 3405 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ccb9bfbd91334a59bb9b562bc5be7fa6: acquiring CA information for follower catalog manager: success
I20260812 01:19:26.407639 3405 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ccb9bfbd91334a59bb9b562bc5be7fa6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 01:19:26.454206 3408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:26.447466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54972 (local address 127.24.0.188:37741)
0812 01:19:26.447628 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:26.447629 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:26.447637 (+ 8us) server_negotiation.cc:407] Connection header received
0812 01:19:26.447661 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:26.447661 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:26.447671 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:26.447750 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:26.448437 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.449105 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.449829 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.450062 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.450144 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:26.451404 (+ 1260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:26.451406 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:26.451406 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:26.451414 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:26.452945 (+ 1531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:26.453366 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:26.453367 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:26.453367 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:26.453427 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:26.453647 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:26.453648 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:26.453648 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:26.453757 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:26.453860 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:26.453970 (+ 110us) server_negotiation.cc:299] Negotiation successful
0812 01:19:26.454065 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":89,"threads_started":1}
I20260812 01:19:26.454779 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 2132
I20260812 01:19:26.468219 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.0.129:34201
--local_ip_for_outbound_sockets=127.24.0.129
--tserver_master_addrs=127.24.0.190:46409
--webserver_port=43523
--webserver_interface=127.24.0.129
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/krb5kdc/oryx_127.24.0.129.keytab
--principal=oryx/127.24.0.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:38987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.24.0.190:46409,127.24.0.189:38795
--tserver_master_addrs=127.24.0.190:46409,127.24.0.189:38795,127.24.0.188:37741 with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:19:26.552109 3410 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:19:26.552335 3410 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:19:26.552359 3410 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:19:26.553534 3410 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:19:26.553577 3410 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:19:26.553603 3410 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.0.129
I20260812 01:19:26.555111 3410 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:38987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/krb5kdc/oryx_127.24.0.129.keytab
--principal=oryx/127.24.0.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.0.129:34201
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.0.129
--webserver_port=43523
--tserver_master_addrs=127.24.0.190:46409,127.24.0.189:38795,127.24.0.188:37741
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3410
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.0.129
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:19:26.555387 3410 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:19:26.555630 3410 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:19:26.560788 3423 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:26.560904 3420 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:26.560788 3418 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:19:26.561048 3410 server_base.cc:1034] running on GCE node
I20260812 01:19:26.561239 3410 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:19:26.561483 3410 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:19:26.562675 3410 hybrid_clock.cc:648] HybridClock initialized: now 1786497566562670 us; error 44 us; skew 500 ppm
Aug 12 01:19:26 dist-test-slave-t3q3 krb5kdc[1963](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497566, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.0.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:19:26.565450 3410 init.cc:377] Logged in from keytab as oryx/127.24.0.129@KRBTEST.COM (short username oryx)
I20260812 01:19:26.565954 3410 webserver.cc:460] Webserver started at http://127.24.0.129:43523/ using document root <none> and password file <none>
I20260812 01:19:26.566205 3410 fs_manager.cc:359] Metadata directory not provided
I20260812 01:19:26.566303 3410 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:19:26.567659 3410 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:19:26.568315 3434 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:26.568487 3410 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:19:26.568593 3410 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/ts-0/wal
uuid: "06e0895a4bc34d5dad8a872a6fe475eb"
format_stamp: "Formatted at 2026-08-12 01:19:25 on dist-test-slave-t3q3"
server_key: "a18da99b60fcbf0543b9a9b15666417e"
server_key_iv: "3ddd2a9371893bda27ac4002ba4eca94"
server_key_version: "encryptionkey@0"
I20260812 01:19:26.568938 3410 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:19:26.586198 3410 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:19:26.588311 3410 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:19:26.588466 3410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:19:26.588697 3410 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:19:26.589010 3410 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:19:26.589076 3410 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:26.589123 3410 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:19:26.589147 3410 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:26.595168 3410 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.129:34201
I20260812 01:19:26.595222 3627 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.129:34201 every 8 connection(s)
I20260812 01:19:26.595607 3410 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/ts-0/data/info.pb
Aug 12 01:19:26 dist-test-slave-t3q3 krb5kdc[1963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497566, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.0.129@KRBTEST.COM for oryx/127.24.0.189@KRBTEST.COM
Aug 12 01:19:26 dist-test-slave-t3q3 krb5kdc[1963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497566, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.0.129@KRBTEST.COM for oryx/127.24.0.188@KRBTEST.COM
Aug 12 01:19:26 dist-test-slave-t3q3 krb5kdc[1963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497566, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.0.129@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM
I20260812 01:19:26.604995 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 3410
I20260812 01:19:26.605623 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 2355
I20260812 01:19:26.607595 3408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:26.596335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.129:38623 (local address 127.24.0.188:37741)
0812 01:19:26.596419 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:26.596422 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:26.596673 (+ 251us) server_negotiation.cc:407] Connection header received
0812 01:19:26.598018 (+ 1345us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:26.598019 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:26.598033 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:26.598066 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:26.598865 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.599624 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.601047 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.601244 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.601293 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:26.604186 (+ 2893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:26.604188 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:26.604188 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:26.604199 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:26.606127 (+ 1928us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:26.606544 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:26.606545 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:26.606546 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:26.606606 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:26.606826 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:26.606827 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:26.606827 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:26.606966 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:26.607012 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:26.607381 (+ 369us) server_negotiation.cc:299] Negotiation successful
0812 01:19:26.607477 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 01:19:26.607669 3267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:26.596353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.129:36183 (local address 127.24.0.189:38795)
0812 01:19:26.596409 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:26.596412 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:26.596875 (+ 463us) server_negotiation.cc:407] Connection header received
0812 01:19:26.597992 (+ 1117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:26.597993 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:26.598009 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:26.598063 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:26.598842 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.599357 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.600426 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.600629 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.600721 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:26.603609 (+ 2888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:26.603611 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:26.603611 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:26.603618 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:26.605657 (+ 2039us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:26.606207 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:26.606209 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:26.606209 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:26.606278 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:26.606650 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:26.606651 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:26.606652 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:26.606848 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:26.606921 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:26.607396 (+ 475us) server_negotiation.cc:299] Negotiation successful
0812 01:19:26.607516 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 01:19:26.608215 3638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:26.596354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.188:37741 (local address 127.24.0.129:38623)
0812 01:19:26.596534 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:26.596575 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:26.597887 (+ 1312us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:26.598141 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:26.598142 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:26.598542 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:26.598751 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:26.598757 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.599728 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.599729 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:26.600678 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:26.600680 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.601322 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.601323 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:26.601387 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:26.601784 (+ 397us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:26.601789 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:26.604039 (+ 2250us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:19:26.606209 (+ 2170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:26.606210 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:26.606211 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:26.606446 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:26.606688 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:26.606689 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:26.606689 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:26.606750 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:26.607062 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:19:26.607063 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:19:26.607210 (+ 147us) client_negotiation.cc:769] Sending connection context
0812 01:19:26.607726 (+ 516us) client_negotiation.cc:240] Negotiation successful
0812 01:19:26.607931 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"mutex_wait_us":53,"thread_start_us":103,"threads_started":1}
I20260812 01:19:26.608309 3640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:26.596355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.189:38795 (local address 127.24.0.129:36183)
0812 01:19:26.596807 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:26.596819 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:26.597890 (+ 1071us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:26.598135 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:26.598139 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:26.598533 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:26.598749 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:26.598754 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.599466 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.599467 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:26.600340 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:26.600343 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.601368 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.601370 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:26.601437 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:26.601939 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:26.601944 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:26.603487 (+ 1543us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:19:26.605754 (+ 2267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:26.605761 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:26.605763 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:26.606107 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:26.606357 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:26.606357 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:26.606358 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:26.606549 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:26.606994 (+ 445us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:19:26.607003 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:19:26.607213 (+ 210us) client_negotiation.cc:769] Sending connection context
0812 01:19:26.607772 (+ 559us) client_negotiation.cc:240] Negotiation successful
0812 01:19:26.608048 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":424,"mutex_wait_us":118,"thread_start_us":110,"threads_started":1}
I20260812 01:19:26.608413 2108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:26.596789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.129:37797 (local address 127.24.0.190:46409)
0812 01:19:26.596841 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:26.596843 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:26.596954 (+ 111us) server_negotiation.cc:407] Connection header received
0812 01:19:26.598018 (+ 1064us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:26.598019 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:26.598033 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:26.598066 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:26.598865 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.599629 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.600747 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.600945 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.600998 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:26.604829 (+ 3831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:26.604831 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:26.604831 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:26.604841 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:26.607085 (+ 2244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:26.607463 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:26.607465 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:26.607465 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:26.607523 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:26.607806 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:26.607807 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:26.607807 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:26.607949 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:26.607998 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:26.608232 (+ 234us) server_negotiation.cc:299] Negotiation successful
0812 01:19:26.608323 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 01:19:26.608800 3631 heartbeater.cc:347] Connected to a master server at 127.24.0.189:38795
I20260812 01:19:26.608891 3631 heartbeater.cc:464] Registering TS with master...
I20260812 01:19:26.608908 3637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:26.596659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:46409 (local address 127.24.0.129:37797)
0812 01:19:26.596901 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:26.596910 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:26.597888 (+ 978us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:26.598143 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:26.598143 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:26.598574 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:26.598751 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:26.598757 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.599728 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.599729 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:26.600670 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:26.600672 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.601023 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.601023 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:26.601128 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:26.601710 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:26.601721 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:26.604717 (+ 2996us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:19:26.607165 (+ 2448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:26.607166 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:26.607166 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:26.607381 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:26.607595 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:26.607596 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:26.607596 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:26.607657 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:26.608048 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:19:26.608049 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:19:26.608110 (+ 61us) client_negotiation.cc:769] Sending connection context
0812 01:19:26.608504 (+ 394us) client_negotiation.cc:240] Negotiation successful
0812 01:19:26.608609 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215,"mutex_wait_us":50,"thread_start_us":65,"threads_started":1}
I20260812 01:19:26.609107 3631 heartbeater.cc:511] Master 127.24.0.189:38795 requested a full tablet report, sending...
I20260812 01:19:26.609318 3630 heartbeater.cc:347] Connected to a master server at 127.24.0.190:46409
I20260812 01:19:26.609360 3630 heartbeater.cc:464] Registering TS with master...
I20260812 01:19:26.609436 3630 heartbeater.cc:511] Master 127.24.0.190:46409 requested a full tablet report, sending...
I20260812 01:19:26.609723 3031 ts_manager.cc:194] Registered new tserver with Master: 06e0895a4bc34d5dad8a872a6fe475eb (127.24.0.129:34201)
I20260812 01:19:26.609777 2035 ts_manager.cc:194] Re-registered known tserver with Master: 06e0895a4bc34d5dad8a872a6fe475eb (127.24.0.129:34201)
I20260812 01:19:26.610445 2035 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.0.129@KRBTEST.COM'} at 127.24.0.129:37797
I20260812 01:19:26.615641 3628 heartbeater.cc:347] Connected to a master server at 127.24.0.188:37741
I20260812 01:19:26.615695 3628 heartbeater.cc:464] Registering TS with master...
I20260812 01:19:26.615772 3628 heartbeater.cc:511] Master 127.24.0.188:37741 requested a full tablet report, sending...
I20260812 01:19:26.616233 3327 ts_manager.cc:194] Registered new tserver with Master: 06e0895a4bc34d5dad8a872a6fe475eb (127.24.0.129:34201)
I20260812 01:19:26.619861 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.0.130:45783
--local_ip_for_outbound_sockets=127.24.0.130
--tserver_master_addrs=127.24.0.190:46409
--webserver_port=36465
--webserver_interface=127.24.0.130
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/krb5kdc/oryx_127.24.0.130.keytab
--principal=oryx/127.24.0.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:38987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.24.0.190:46409,127.24.0.189:38795
--tserver_master_addrs=127.24.0.190:46409,127.24.0.189:38795,127.24.0.188:37741 with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:19:26.705426 3641 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:19:26.705644 3641 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:19:26.705668 3641 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:19:26.706764 3641 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:19:26.706799 3641 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:19:26.706825 3641 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.0.130
I20260812 01:19:26.708282 3641 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:38987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/krb5kdc/oryx_127.24.0.130.keytab
--principal=oryx/127.24.0.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.0.130:45783
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.0.130
--webserver_port=36465
--tserver_master_addrs=127.24.0.190:46409,127.24.0.189:38795,127.24.0.188:37741
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3641
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.0.130
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:19:26.708489 3641 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:19:26.708673 3641 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:19:26.713730 3651 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:26.713742 3648 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:26.713742 3654 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:19:26.714013 3641 server_base.cc:1034] running on GCE node
I20260812 01:19:26.714229 3641 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:19:26.714474 3641 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:19:26.715623 3641 hybrid_clock.cc:648] HybridClock initialized: now 1786497566715607 us; error 30 us; skew 500 ppm
Aug 12 01:19:26 dist-test-slave-t3q3 krb5kdc[1963](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497566, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.0.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:19:26.718232 3641 init.cc:377] Logged in from keytab as oryx/127.24.0.130@KRBTEST.COM (short username oryx)
I20260812 01:19:26.718621 3641 webserver.cc:460] Webserver started at http://127.24.0.130:36465/ using document root <none> and password file <none>
I20260812 01:19:26.718824 3641 fs_manager.cc:359] Metadata directory not provided
I20260812 01:19:26.718883 3641 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:19:26.720232 3641 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 01:19:26.720969 3664 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:26.721174 3641 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 01:19:26.721271 3641 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/ts-1/wal
uuid: "d645f2e569134a37baea2860781c5e0e"
format_stamp: "Formatted at 2026-08-12 01:19:25 on dist-test-slave-t3q3"
server_key: "d04f40c4310226eea803ec7ea8dc7df1"
server_key_iv: "199e0cdc11be2afe4ca45f67eb1d48a7"
server_key_version: "encryptionkey@0"
I20260812 01:19:26.721611 3641 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:19:26.734939 3641 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:19:26.738123 3641 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:19:26.738312 3641 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:19:26.738564 3641 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:19:26.738893 3641 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:19:26.738948 3641 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:26.738992 3641 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:19:26.739020 3641 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:26.745042 3641 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.130:45783
I20260812 01:19:26.745097 3866 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.130:45783 every 8 connection(s)
I20260812 01:19:26.745518 3641 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/ts-1/data/info.pb
Aug 12 01:19:26 dist-test-slave-t3q3 krb5kdc[1963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497566, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.0.130@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM
Aug 12 01:19:26 dist-test-slave-t3q3 krb5kdc[1963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497566, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.0.130@KRBTEST.COM for oryx/127.24.0.188@KRBTEST.COM
Aug 12 01:19:26 dist-test-slave-t3q3 krb5kdc[1963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497566, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.0.130@KRBTEST.COM for oryx/127.24.0.189@KRBTEST.COM
I20260812 01:19:26.755419 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 3641
I20260812 01:19:26.755576 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 2585
I20260812 01:19:26.757865 2108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:26.746727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.130:33521 (local address 127.24.0.190:46409)
0812 01:19:26.746769 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:26.746771 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:26.747157 (+ 386us) server_negotiation.cc:407] Connection header received
0812 01:19:26.747571 (+ 414us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:26.747572 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:26.747584 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:26.747652 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:26.748424 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.748933 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.749948 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.750164 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.750243 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:26.753816 (+ 3573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:26.753818 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:26.753818 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:26.753826 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:26.755511 (+ 1685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:26.757096 (+ 1585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:26.757097 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:26.757097 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:26.757152 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:26.757322 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:26.757322 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:26.757322 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:26.757436 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:26.757479 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:26.757659 (+ 180us) server_negotiation.cc:299] Negotiation successful
0812 01:19:26.757737 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 01:19:26.758168 3408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:26.746867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.130:51823 (local address 127.24.0.188:37741)
0812 01:19:26.746913 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:26.746915 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:26.746927 (+ 12us) server_negotiation.cc:407] Connection header received
0812 01:19:26.747440 (+ 513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:26.747441 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:26.747455 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:26.747505 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:26.748447 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.749212 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.750366 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.750662 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.750723 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:26.754117 (+ 3394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:26.754118 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:26.754119 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:26.754126 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:26.755878 (+ 1752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:26.756288 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:26.756290 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:26.756290 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:26.756341 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:26.756593 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:26.756594 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:26.756594 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:26.756722 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:26.756768 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:26.758046 (+ 1278us) server_negotiation.cc:299] Negotiation successful
0812 01:19:26.758111 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 01:19:26.758694 3875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:26.746667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:46409 (local address 127.24.0.130:33521)
0812 01:19:26.747087 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:26.747096 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:26.747335 (+ 239us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:26.747672 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:26.747673 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:26.748036 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:26.748325 (+ 289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:26.748329 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.749043 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.749044 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:26.749837 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:26.749839 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.750251 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.750252 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:26.750396 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:26.750986 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:26.750997 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:26.753719 (+ 2722us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:19:26.755620 (+ 1901us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:26.755627 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:26.755630 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:26.755931 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:26.757216 (+ 1285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:26.757216 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:26.757216 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:26.757268 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:26.757504 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:19:26.757505 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:19:26.757537 (+ 32us) client_negotiation.cc:769] Sending connection context
0812 01:19:26.758273 (+ 736us) client_negotiation.cc:240] Negotiation successful
0812 01:19:26.758472 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":83,"threads_started":1}
I20260812 01:19:26.758984 3874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:26.746380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.188:37741 (local address 127.24.0.130:51823)
0812 01:19:26.746575 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:26.746620 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:26.747326 (+ 706us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:26.747577 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:26.747580 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:26.748193 (+ 613us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:26.748325 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:26.748331 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.749312 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.749312 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:26.750269 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:26.750271 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.750744 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.750745 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:26.750823 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:26.751273 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:26.751278 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:26.754046 (+ 2768us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:19:26.755999 (+ 1953us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:26.756000 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:26.756000 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:26.756213 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:26.756403 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:26.756404 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:26.756404 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:26.756521 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:26.756829 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:19:26.756835 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:19:26.756979 (+ 144us) client_negotiation.cc:769] Sending connection context
0812 01:19:26.758767 (+ 1788us) client_negotiation.cc:240] Negotiation successful
0812 01:19:26.758856 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"mutex_wait_us":36,"thread_start_us":93,"threads_started":1}
I20260812 01:19:26.771626 3868 heartbeater.cc:347] Connected to a master server at 127.24.0.190:46409
I20260812 01:19:26.771675 3867 heartbeater.cc:347] Connected to a master server at 127.24.0.188:37741
I20260812 01:19:26.771766 3868 heartbeater.cc:464] Registering TS with master...
I20260812 01:19:26.771787 3867 heartbeater.cc:464] Registering TS with master...
I20260812 01:19:26.771981 3868 heartbeater.cc:511] Master 127.24.0.190:46409 requested a full tablet report, sending...
I20260812 01:19:26.771981 3867 heartbeater.cc:511] Master 127.24.0.188:37741 requested a full tablet report, sending...
I20260812 01:19:26.772425 2035 ts_manager.cc:194] Re-registered known tserver with Master: d645f2e569134a37baea2860781c5e0e (127.24.0.130:45783)
I20260812 01:19:26.772486 3327 ts_manager.cc:194] Registered new tserver with Master: d645f2e569134a37baea2860781c5e0e (127.24.0.130:45783)
I20260812 01:19:26.772538 3267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:26.746851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.130:60179 (local address 127.24.0.189:38795)
0812 01:19:26.746909 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:26.746911 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:26.747045 (+ 134us) server_negotiation.cc:407] Connection header received
0812 01:19:26.747712 (+ 667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:26.747714 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:26.747726 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:26.747780 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:26.748446 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.749207 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.750771 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.751116 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.751190 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:26.753843 (+ 2653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:26.753844 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:26.753844 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:26.753852 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:26.755803 (+ 1951us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:26.771509 (+ 15706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:26.771511 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:26.771511 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:26.771595 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:26.771825 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:26.771826 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:26.771826 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:26.771975 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:26.772028 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:26.772292 (+ 264us) server_negotiation.cc:299] Negotiation successful
0812 01:19:26.772401 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 01:19:26.772959 3876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:26.746383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.189:38795 (local address 127.24.0.130:60179)
0812 01:19:26.746972 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:26.746984 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:26.747327 (+ 343us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:26.747806 (+ 479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:26.747807 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:26.748129 (+ 322us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:26.748325 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:26.748331 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.749312 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.749313 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:26.750269 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:26.750271 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.751247 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.751248 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:26.751312 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:26.751690 (+ 378us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:26.751696 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:26.753751 (+ 2055us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:19:26.771114 (+ 17363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:26.771115 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:26.771116 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:26.771403 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:26.771684 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:26.771685 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:26.771685 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:26.771753 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:26.772121 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:19:26.772122 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:19:26.772170 (+ 48us) client_negotiation.cc:769] Sending connection context
0812 01:19:26.772662 (+ 492us) client_negotiation.cc:240] Negotiation successful
0812 01:19:26.772784 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":569,"mutex_wait_us":69,"spinlock_wait_cycles":384,"thread_start_us":120,"threads_started":1}
I20260812 01:19:26.772954 2035 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.0.130@KRBTEST.COM'} at 127.24.0.130:33521
I20260812 01:19:26.773499 3870 heartbeater.cc:347] Connected to a master server at 127.24.0.189:38795
I20260812 01:19:26.773557 3870 heartbeater.cc:464] Registering TS with master...
I20260812 01:19:26.773752 3870 heartbeater.cc:511] Master 127.24.0.189:38795 requested a full tablet report, sending...
I20260812 01:19:26.774055 3031 ts_manager.cc:194] Registered new tserver with Master: d645f2e569134a37baea2860781c5e0e (127.24.0.130:45783)
I20260812 01:19:26.776101 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.0.131:41837
--local_ip_for_outbound_sockets=127.24.0.131
--tserver_master_addrs=127.24.0.190:46409
--webserver_port=43941
--webserver_interface=127.24.0.131
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/krb5kdc/oryx_127.24.0.131.keytab
--principal=oryx/127.24.0.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:38987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.24.0.190:46409,127.24.0.189:38795
--tserver_master_addrs=127.24.0.190:46409,127.24.0.189:38795,127.24.0.188:37741 with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:19:26.860459 3877 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:19:26.860656 3877 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:19:26.860716 3877 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:19:26.861835 3877 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:19:26.861876 3877 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:19:26.861902 3877 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.0.131
I20260812 01:19:26.863302 3877 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:38987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/krb5kdc/oryx_127.24.0.131.keytab
--principal=oryx/127.24.0.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.0.131:41837
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.0.131
--webserver_port=43941
--tserver_master_addrs=127.24.0.190:46409,127.24.0.189:38795,127.24.0.188:37741
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3877
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.0.131
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:19:26.863492 3877 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:19:26.863672 3877 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:19:26.868716 3890 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:26.868716 3891 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:19:26.868885 3877 server_base.cc:1034] running on GCE node
W20260812 01:19:26.868711 3894 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:19:26.869171 3877 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:19:26.869458 3877 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:19:26.870651 3877 hybrid_clock.cc:648] HybridClock initialized: now 1786497566870626 us; error 34 us; skew 500 ppm
Aug 12 01:19:26 dist-test-slave-t3q3 krb5kdc[1963](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497566, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.0.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:19:26.873454 3877 init.cc:377] Logged in from keytab as oryx/127.24.0.131@KRBTEST.COM (short username oryx)
I20260812 01:19:26.873867 3877 webserver.cc:460] Webserver started at http://127.24.0.131:43941/ using document root <none> and password file <none>
I20260812 01:19:26.874100 3877 fs_manager.cc:359] Metadata directory not provided
I20260812 01:19:26.874176 3877 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:19:26.875638 3877 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 01:19:26.876376 3908 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:26.876559 3877 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:19:26.876649 3877 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/ts-2/wal
uuid: "85a369874676488a8c43c1a2cf51a95a"
format_stamp: "Formatted at 2026-08-12 01:19:25 on dist-test-slave-t3q3"
server_key: "076ef42558e5affa82a5a770341874dd"
server_key_iv: "f052eb34a4dcf94de07288d3f4483747"
server_key_version: "encryptionkey@0"
I20260812 01:19:26.876967 3877 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:19:26.886629 3877 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:19:26.889096 3877 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:19:26.889220 3877 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:19:26.889420 3877 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:19:26.889756 3877 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:19:26.889793 3877 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:26.889816 3877 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:19:26.889829 3877 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:26.896574 3877 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.131:41837
I20260812 01:19:26.896625 4096 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.131:41837 every 8 connection(s)
I20260812 01:19:26.896991 3877 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/ts-2/data/info.pb
I20260812 01:19:26.901429 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 3877
Aug 12 01:19:26 dist-test-slave-t3q3 krb5kdc[1963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497566, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.0.131@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM
Aug 12 01:19:26 dist-test-slave-t3q3 krb5kdc[1963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497566, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.0.131@KRBTEST.COM for oryx/127.24.0.189@KRBTEST.COM
Aug 12 01:19:26 dist-test-slave-t3q3 krb5kdc[1963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497566, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.0.131@KRBTEST.COM for oryx/127.24.0.188@KRBTEST.COM
I20260812 01:19:26.909727 2108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:26.898280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.131:35129 (local address 127.24.0.190:46409)
0812 01:19:26.898384 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:26.898386 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:26.898395 (+ 9us) server_negotiation.cc:407] Connection header received
0812 01:19:26.898803 (+ 408us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:26.898804 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:26.898816 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:26.898876 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:26.899835 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.900632 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.901839 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.902253 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.902304 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:26.906409 (+ 4105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:26.906411 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:26.906411 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:26.906419 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:26.908003 (+ 1584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:26.908626 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:26.908628 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:26.908628 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:26.908694 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:26.908975 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:26.908976 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:26.908976 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:26.909116 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:26.909158 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:26.909541 (+ 383us) server_negotiation.cc:299] Negotiation successful
0812 01:19:26.909627 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260812 01:19:26.910074 4106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:26.898038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.189:38795 (local address 127.24.0.131:48567)
0812 01:19:26.898206 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:26.898233 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:26.898571 (+ 338us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:26.899055 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:26.899056 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:26.899499 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:26.899711 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:26.899718 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.900738 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.900738 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:26.901740 (+ 1002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:26.901742 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.902487 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.902488 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:26.902559 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:26.902921 (+ 362us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:26.902925 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:26.905755 (+ 2830us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:19:26.908273 (+ 2518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:26.908274 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:26.908275 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:26.908885 (+ 610us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:26.909111 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:26.909112 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:26.909112 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:26.909175 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:26.909504 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:19:26.909505 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:19:26.909546 (+ 41us) client_negotiation.cc:769] Sending connection context
0812 01:19:26.909624 (+ 78us) client_negotiation.cc:240] Negotiation successful
0812 01:19:26.909812 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"mutex_wait_us":42,"thread_start_us":103,"threads_started":1}
I20260812 01:19:26.910074 4104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:26.897727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:46409 (local address 127.24.0.131:35129)
0812 01:19:26.897901 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:26.897943 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:26.898563 (+ 620us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:26.898921 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:26.898923 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:26.899500 (+ 577us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:26.899712 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:26.899718 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.900738 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.900738 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:26.901732 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:26.901735 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.902394 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.902395 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:26.902467 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:26.902839 (+ 372us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:26.902852 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:26.906297 (+ 3445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:19:26.908147 (+ 1850us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:26.908154 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:26.908156 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:26.908536 (+ 380us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:26.908768 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:26.908769 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:26.908769 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:26.908899 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:26.909203 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:19:26.909209 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:19:26.909382 (+ 173us) client_negotiation.cc:769] Sending connection context
0812 01:19:26.909819 (+ 437us) client_negotiation.cc:240] Negotiation successful
0812 01:19:26.909930 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"mutex_wait_us":45,"thread_start_us":158,"threads_started":1}
I20260812 01:19:26.911402 4101 heartbeater.cc:347] Connected to a master server at 127.24.0.190:46409
I20260812 01:19:26.911434 3267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:26.898321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.131:48567 (local address 127.24.0.189:38795)
0812 01:19:26.898369 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:26.898371 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:26.898381 (+ 10us) server_negotiation.cc:407] Connection header received
0812 01:19:26.898942 (+ 561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:26.898943 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:26.898956 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:26.899018 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:26.899835 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.900632 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.901838 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.902271 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.902310 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:26.905878 (+ 3568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:26.905880 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:26.905880 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:26.905888 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:26.908184 (+ 2296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:26.908975 (+ 791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:26.908977 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:26.908977 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:26.909036 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:26.909267 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:26.909268 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:26.909269 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:26.909413 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:26.909478 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:26.911230 (+ 1752us) server_negotiation.cc:299] Negotiation successful
0812 01:19:26.911316 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 01:19:26.911517 4101 heartbeater.cc:464] Registering TS with master...
I20260812 01:19:26.911729 4102 heartbeater.cc:347] Connected to a master server at 127.24.0.189:38795
I20260812 01:19:26.911772 4101 heartbeater.cc:511] Master 127.24.0.190:46409 requested a full tablet report, sending...
I20260812 01:19:26.911774 4102 heartbeater.cc:464] Registering TS with master...
I20260812 01:19:26.911983 4102 heartbeater.cc:511] Master 127.24.0.189:38795 requested a full tablet report, sending...
I20260812 01:19:26.912524 2035 ts_manager.cc:194] Re-registered known tserver with Master: 85a369874676488a8c43c1a2cf51a95a (127.24.0.131:41837)
I20260812 01:19:26.912597 3031 ts_manager.cc:194] Registered new tserver with Master: 85a369874676488a8c43c1a2cf51a95a (127.24.0.131:41837)
I20260812 01:19:26.913036 2035 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.0.131@KRBTEST.COM'} at 127.24.0.131:35129
I20260812 01:19:26.913705 4117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:26.903328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60114 (local address 127.24.0.190:46409)
0812 01:19:26.904316 (+ 988us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:26.904318 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:26.904444 (+ 126us) server_negotiation.cc:407] Connection header received
0812 01:19:26.904721 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:26.904722 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:26.904738 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:26.904818 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:26.905838 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.906600 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.907758 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.908117 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.908183 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:26.909731 (+ 1548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:26.909732 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:26.909733 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:26.909740 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:26.912143 (+ 2403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:26.912591 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:26.912592 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:26.912592 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:26.912652 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:26.912879 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:26.912879 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:26.912880 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:26.913034 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:26.913130 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:26.913465 (+ 335us) server_negotiation.cc:299] Negotiation successful
0812 01:19:26.913578 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":961,"thread_start_us":49,"threads_started":1}
I20260812 01:19:26.916344 4122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:26.904124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44136 (local address 127.24.0.189:38795)
0812 01:19:26.904254 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:26.904256 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:26.904855 (+ 599us) server_negotiation.cc:407] Connection header received
0812 01:19:26.904879 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:26.904880 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:26.904891 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:26.904958 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:26.906664 (+ 1706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.907426 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.910346 (+ 2920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.910649 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.910712 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:26.912735 (+ 2023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:26.912736 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:26.912737 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:26.912743 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:26.914739 (+ 1996us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:26.915166 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:26.915168 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:26.915168 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:26.915227 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:26.915429 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:26.915430 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:26.915431 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:26.915559 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:26.915658 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:26.916141 (+ 483us) server_negotiation.cc:299] Negotiation successful
0812 01:19:26.916233 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":60,"threads_started":1}
I20260812 01:19:26.916673 4105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:26.897727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.188:37741 (local address 127.24.0.131:51307)
0812 01:19:26.897952 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:26.897963 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:26.898554 (+ 591us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:26.898811 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:26.898815 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:26.899491 (+ 676us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:26.899710 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:26.899715 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.900495 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.900496 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:26.901423 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:26.901426 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.901985 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.901985 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:26.902191 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:26.903176 (+ 985us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:26.903182 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:26.913303 (+ 10121us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:19:26.915540 (+ 2237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:26.915541 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:26.915541 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:26.915769 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:26.915987 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:26.915987 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:26.915988 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:26.916039 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:26.916352 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:19:26.916353 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:19:26.916392 (+ 39us) client_negotiation.cc:769] Sending connection context
0812 01:19:26.916463 (+ 71us) client_negotiation.cc:240] Negotiation successful
0812 01:19:26.916571 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"mutex_wait_us":2,"thread_start_us":95,"threads_started":1}
I20260812 01:19:26.916709 3408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:26.897729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.131:51307 (local address 127.24.0.188:37741)
0812 01:19:26.897800 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:26.897803 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:26.898030 (+ 227us) server_negotiation.cc:407] Connection header received
0812 01:19:26.898668 (+ 638us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:26.898670 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:26.898683 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:26.898739 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:26.899809 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.900389 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.901539 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.901840 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.901886 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:26.913402 (+ 11516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:26.913403 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:26.913404 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:26.913412 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:26.915454 (+ 2042us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:26.915859 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:26.915860 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:26.915861 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:26.915917 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:26.916139 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:26.916140 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:26.916140 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:26.916260 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:26.916304 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:26.916506 (+ 202us) server_negotiation.cc:299] Negotiation successful
0812 01:19:26.916587 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 01:19:26.917085 4100 heartbeater.cc:347] Connected to a master server at 127.24.0.188:37741
I20260812 01:19:26.917137 4100 heartbeater.cc:464] Registering TS with master...
I20260812 01:19:26.917212 4100 heartbeater.cc:511] Master 127.24.0.188:37741 requested a full tablet report, sending...
I20260812 01:19:26.917572 3327 ts_manager.cc:194] Registered new tserver with Master: 85a369874676488a8c43c1a2cf51a95a (127.24.0.131:41837)
I20260812 01:19:26.915704 2035 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60114:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 01:19:26.918099 2035 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 01:19:26.918671 4116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:26.903143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54984 (local address 127.24.0.188:37741)
0812 01:19:26.905188 (+ 2045us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:26.905190 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:26.905198 (+ 8us) server_negotiation.cc:407] Connection header received
0812 01:19:26.905218 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:26.905219 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:26.905229 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:26.905298 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:26.907530 (+ 2232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.908302 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.910373 (+ 2071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.910677 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.910732 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:26.915377 (+ 4645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:26.915379 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:26.915379 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:26.915387 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:26.917577 (+ 2190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:26.917956 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:26.917957 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:26.917957 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:26.918011 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:26.918215 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:26.918215 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:26.918215 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:26.918303 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:26.918370 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:26.918502 (+ 132us) server_negotiation.cc:299] Negotiation successful
0812 01:19:26.918578 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2025,"thread_start_us":56,"threads_started":1}
I20260812 01:19:26.926205 4135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:26.922032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.129:34201 (local address 127.0.0.1:33812)
0812 01:19:26.922216 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:26.922227 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:26.922301 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:26.922614 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:26.922615 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:26.922617 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:19:26.922686 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:26.922688 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.923872 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.923873 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:26.925083 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:26.925085 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.925949 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.925950 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:26.926011 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:26.926017 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 01:19:26.926079 (+ 62us) client_negotiation.cc:240] Negotiation successful
0812 01:19:26.926120 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":80,"threads_started":1}
I20260812 01:19:26.926205 4136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:26.922138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33812 (local address 127.24.0.129:34201)
0812 01:19:26.922408 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:26.922413 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:26.922425 (+ 12us) server_negotiation.cc:407] Connection header received
0812 01:19:26.922470 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:26.922472 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:26.922491 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:26.922591 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:19:26.922771 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.923791 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.925171 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.925844 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.925945 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:26.926025 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:26.926094 (+ 69us) server_negotiation.cc:299] Negotiation successful
0812 01:19:26.926131 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":228,"thread_start_us":186,"threads_started":1}
I20260812 01:19:26.926641 4137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:26.922304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59774 (local address 127.24.0.131:41837)
0812 01:19:26.922683 (+ 379us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:26.922688 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:26.922697 (+ 9us) server_negotiation.cc:407] Connection header received
0812 01:19:26.922757 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:26.922759 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:26.922775 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:26.922863 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:19:26.922975 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.924095 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.925568 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.926294 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.926364 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:26.926434 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:26.926517 (+ 83us) server_negotiation.cc:299] Negotiation successful
0812 01:19:26.926564 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":81,"threads_started":1}
I20260812 01:19:26.926607 4134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:26.922078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.131:41837 (local address 127.0.0.1:59774)
0812 01:19:26.922386 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:26.922395 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:26.922433 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:26.922869 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:26.922870 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:26.922872 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:19:26.922900 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:26.922901 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.924192 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.924193 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:26.925480 (+ 1287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:26.925483 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.926368 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.926369 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:26.926436 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:26.926441 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 01:19:26.926503 (+ 62us) client_negotiation.cc:240] Negotiation successful
0812 01:19:26.926539 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":99,"threads_started":1}
I20260812 01:19:26.927381 4132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:26.922087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.130:45783 (local address 127.0.0.1:32896)
0812 01:19:26.923033 (+ 946us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:26.923044 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:26.923107 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:26.923287 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:26.923288 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:26.923290 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:19:26.923334 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:26.923335 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.924932 (+ 1597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.924933 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:26.926040 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:26.926053 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.927141 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.927142 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:26.927212 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:26.927216 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 01:19:26.927271 (+ 55us) client_negotiation.cc:240] Negotiation successful
0812 01:19:26.927314 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":926,"thread_start_us":89,"threads_started":1}
I20260812 01:19:26.927430 4138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:26.922470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32896 (local address 127.24.0.130:45783)
0812 01:19:26.922980 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:26.922984 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:26.923126 (+ 142us) server_negotiation.cc:407] Connection header received
0812 01:19:26.923174 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:26.923176 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:26.923193 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:26.923277 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:19:26.923391 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.924833 (+ 1442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.926159 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.926887 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.926947 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:26.927021 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:26.927301 (+ 280us) server_negotiation.cc:299] Negotiation successful
0812 01:19:26.927345 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":473,"thread_start_us":86,"threads_started":1}
I20260812 01:19:26.927386 3516 tablet_service.cc:1467] Processing CreateTablet for tablet 15759fb12e804b76989e2a1a10b9361b (DEFAULT_TABLE table=test-table [id=b69e85dfa704495e8ead72b3a6d2ae3c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:19:26.927544 3989 tablet_service.cc:1467] Processing CreateTablet for tablet 15759fb12e804b76989e2a1a10b9361b (DEFAULT_TABLE table=test-table [id=b69e85dfa704495e8ead72b3a6d2ae3c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:19:26.927796 3516 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15759fb12e804b76989e2a1a10b9361b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:19:26.927815 3989 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15759fb12e804b76989e2a1a10b9361b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:19:26.928112 3754 tablet_service.cc:1467] Processing CreateTablet for tablet 15759fb12e804b76989e2a1a10b9361b (DEFAULT_TABLE table=test-table [id=b69e85dfa704495e8ead72b3a6d2ae3c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:19:26.928390 3754 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15759fb12e804b76989e2a1a10b9361b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:19:26.932201 4141 tablet_bootstrap.cc:492] T 15759fb12e804b76989e2a1a10b9361b P d645f2e569134a37baea2860781c5e0e: Bootstrap starting.
I20260812 01:19:26.932879 4141 tablet_bootstrap.cc:654] T 15759fb12e804b76989e2a1a10b9361b P d645f2e569134a37baea2860781c5e0e: Neither blocks nor log segments found. Creating new log.
I20260812 01:19:26.933202 4141 log.cc:824] T 15759fb12e804b76989e2a1a10b9361b P d645f2e569134a37baea2860781c5e0e: Log is configured to *not* fsync() on all Append() calls
I20260812 01:19:26.933357 4142 tablet_bootstrap.cc:492] T 15759fb12e804b76989e2a1a10b9361b P 85a369874676488a8c43c1a2cf51a95a: Bootstrap starting.
I20260812 01:19:26.933578 4140 tablet_bootstrap.cc:492] T 15759fb12e804b76989e2a1a10b9361b P 06e0895a4bc34d5dad8a872a6fe475eb: Bootstrap starting.
I20260812 01:19:26.934007 4142 tablet_bootstrap.cc:654] T 15759fb12e804b76989e2a1a10b9361b P 85a369874676488a8c43c1a2cf51a95a: Neither blocks nor log segments found. Creating new log.
I20260812 01:19:26.934160 4140 tablet_bootstrap.cc:654] T 15759fb12e804b76989e2a1a10b9361b P 06e0895a4bc34d5dad8a872a6fe475eb: Neither blocks nor log segments found. Creating new log.
I20260812 01:19:26.934345 4142 log.cc:824] T 15759fb12e804b76989e2a1a10b9361b P 85a369874676488a8c43c1a2cf51a95a: Log is configured to *not* fsync() on all Append() calls
I20260812 01:19:26.934530 4140 log.cc:824] T 15759fb12e804b76989e2a1a10b9361b P 06e0895a4bc34d5dad8a872a6fe475eb: Log is configured to *not* fsync() on all Append() calls
I20260812 01:19:26.935112 4141 tablet_bootstrap.cc:492] T 15759fb12e804b76989e2a1a10b9361b P d645f2e569134a37baea2860781c5e0e: No bootstrap required, opened a new log
I20260812 01:19:26.935186 4142 tablet_bootstrap.cc:492] T 15759fb12e804b76989e2a1a10b9361b P 85a369874676488a8c43c1a2cf51a95a: No bootstrap required, opened a new log
I20260812 01:19:26.935201 4141 ts_tablet_manager.cc:1397] T 15759fb12e804b76989e2a1a10b9361b P d645f2e569134a37baea2860781c5e0e: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260812 01:19:26.935258 4142 ts_tablet_manager.cc:1397] T 15759fb12e804b76989e2a1a10b9361b P 85a369874676488a8c43c1a2cf51a95a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 01:19:26.935362 4140 tablet_bootstrap.cc:492] T 15759fb12e804b76989e2a1a10b9361b P 06e0895a4bc34d5dad8a872a6fe475eb: No bootstrap required, opened a new log
I20260812 01:19:26.935424 4140 ts_tablet_manager.cc:1397] T 15759fb12e804b76989e2a1a10b9361b P 06e0895a4bc34d5dad8a872a6fe475eb: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 01:19:26.936806 4141 raft_consensus.cc:348] T 15759fb12e804b76989e2a1a10b9361b P d645f2e569134a37baea2860781c5e0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06e0895a4bc34d5dad8a872a6fe475eb" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 34201 } } peers { permanent_uuid: "85a369874676488a8c43c1a2cf51a95a" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 41837 } } peers { permanent_uuid: "d645f2e569134a37baea2860781c5e0e" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 45783 } }
I20260812 01:19:26.936967 4141 raft_consensus.cc:374] T 15759fb12e804b76989e2a1a10b9361b P d645f2e569134a37baea2860781c5e0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:19:26.937011 4141 raft_consensus.cc:729] T 15759fb12e804b76989e2a1a10b9361b P d645f2e569134a37baea2860781c5e0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d645f2e569134a37baea2860781c5e0e, State: Initialized, Role: FOLLOWER
I20260812 01:19:26.937144 4141 consensus_queue.cc:260] T 15759fb12e804b76989e2a1a10b9361b P d645f2e569134a37baea2860781c5e0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06e0895a4bc34d5dad8a872a6fe475eb" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 34201 } } peers { permanent_uuid: "85a369874676488a8c43c1a2cf51a95a" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 41837 } } peers { permanent_uuid: "d645f2e569134a37baea2860781c5e0e" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 45783 } }
I20260812 01:19:26.937435 4141 ts_tablet_manager.cc:1428] T 15759fb12e804b76989e2a1a10b9361b P d645f2e569134a37baea2860781c5e0e: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 01:19:26.937341 4142 raft_consensus.cc:348] T 15759fb12e804b76989e2a1a10b9361b P 85a369874676488a8c43c1a2cf51a95a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06e0895a4bc34d5dad8a872a6fe475eb" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 34201 } } peers { permanent_uuid: "85a369874676488a8c43c1a2cf51a95a" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 41837 } } peers { permanent_uuid: "d645f2e569134a37baea2860781c5e0e" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 45783 } }
I20260812 01:19:26.937518 3868 heartbeater.cc:503] Master 127.24.0.190:46409 was elected leader, sending a full tablet report...
I20260812 01:19:26.937521 4142 raft_consensus.cc:374] T 15759fb12e804b76989e2a1a10b9361b P 85a369874676488a8c43c1a2cf51a95a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:19:26.937561 4142 raft_consensus.cc:729] T 15759fb12e804b76989e2a1a10b9361b P 85a369874676488a8c43c1a2cf51a95a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85a369874676488a8c43c1a2cf51a95a, State: Initialized, Role: FOLLOWER
I20260812 01:19:26.937680 4142 consensus_queue.cc:260] T 15759fb12e804b76989e2a1a10b9361b P 85a369874676488a8c43c1a2cf51a95a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06e0895a4bc34d5dad8a872a6fe475eb" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 34201 } } peers { permanent_uuid: "85a369874676488a8c43c1a2cf51a95a" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 41837 } } peers { permanent_uuid: "d645f2e569134a37baea2860781c5e0e" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 45783 } }
I20260812 01:19:26.937942 4142 ts_tablet_manager.cc:1428] T 15759fb12e804b76989e2a1a10b9361b P 85a369874676488a8c43c1a2cf51a95a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 01:19:26.938258 4101 heartbeater.cc:503] Master 127.24.0.190:46409 was elected leader, sending a full tablet report...
I20260812 01:19:26.939441 4140 raft_consensus.cc:348] T 15759fb12e804b76989e2a1a10b9361b P 06e0895a4bc34d5dad8a872a6fe475eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06e0895a4bc34d5dad8a872a6fe475eb" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 34201 } } peers { permanent_uuid: "85a369874676488a8c43c1a2cf51a95a" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 41837 } } peers { permanent_uuid: "d645f2e569134a37baea2860781c5e0e" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 45783 } }
I20260812 01:19:26.939622 4140 raft_consensus.cc:374] T 15759fb12e804b76989e2a1a10b9361b P 06e0895a4bc34d5dad8a872a6fe475eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:19:26.939659 4140 raft_consensus.cc:729] T 15759fb12e804b76989e2a1a10b9361b P 06e0895a4bc34d5dad8a872a6fe475eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06e0895a4bc34d5dad8a872a6fe475eb, State: Initialized, Role: FOLLOWER
I20260812 01:19:26.939764 4140 consensus_queue.cc:260] T 15759fb12e804b76989e2a1a10b9361b P 06e0895a4bc34d5dad8a872a6fe475eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06e0895a4bc34d5dad8a872a6fe475eb" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 34201 } } peers { permanent_uuid: "85a369874676488a8c43c1a2cf51a95a" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 41837 } } peers { permanent_uuid: "d645f2e569134a37baea2860781c5e0e" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 45783 } }
I20260812 01:19:26.940258 3630 heartbeater.cc:503] Master 127.24.0.190:46409 was elected leader, sending a full tablet report...
I20260812 01:19:26.940502 4140 ts_tablet_manager.cc:1428] T 15759fb12e804b76989e2a1a10b9361b P 06e0895a4bc34d5dad8a872a6fe475eb: Time spent starting tablet: real 0.005s user 0.003s sys 0.001s
I20260812 01:19:26.946453 4132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:26.943611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.188:37741 (local address 127.0.0.1:54986)
0812 01:19:26.943695 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:26.943708 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:26.943758 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:26.943927 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:26.943929 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:26.943930 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:19:26.943984 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:26.943986 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.944671 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.944673 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:26.945613 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:26.945615 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.946227 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.946228 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:26.946286 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:26.946289 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 01:19:26.946334 (+ 45us) client_negotiation.cc:240] Negotiation successful
0812 01:19:26.946381 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260812 01:19:26.946453 4116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:26.943716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54986 (local address 127.24.0.188:37741)
0812 01:19:26.943756 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:26.943758 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:26.943767 (+ 9us) server_negotiation.cc:407] Connection header received
0812 01:19:26.943818 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:26.943819 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:26.943849 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:26.943923 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:19:26.944054 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.944583 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.945700 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:26.946139 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:26.946229 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:26.946313 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:26.946339 (+ 26us) server_negotiation.cc:299] Negotiation successful
0812 01:19:26.946381 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 01:19:26.946872 3338 raft_consensus.cc:1203] T 00000000000000000000000000000000 P ccb9bfbd91334a59bb9b562bc5be7fa6 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.9] Dedup: 1.7->[1.8-1.9]
I20260812 01:19:26.947677 4153 sys_catalog.cc:455] T 00000000000000000000000000000000 P ccb9bfbd91334a59bb9b562bc5be7fa6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader efbfb4db2c524879a5d7d17801ff4cb0. Latest consensus state: current_term: 1 leader_uuid: "efbfb4db2c524879a5d7d17801ff4cb0" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "efbfb4db2c524879a5d7d17801ff4cb0" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 46409 } } peers { permanent_uuid: "35a50d99aa1b4d9c8a4b11849067ad0f" member_type: VOTER last_known_addr { host: "127.24.0.189" port: 38795 } attrs { promote: false } } peers { permanent_uuid: "ccb9bfbd91334a59bb9b562bc5be7fa6" member_type: NON_VOTER last_known_addr { host: "127.24.0.188" port: 37741 } attrs { promote: true } } }
I20260812 01:19:26.947790 4153 sys_catalog.cc:458] T 00000000000000000000000000000000 P ccb9bfbd91334a59bb9b562bc5be7fa6 [sys.catalog]: This master's current role is: LEARNER
W20260812 01:19:26.996240 3871 tablet.cc:2369] T 15759fb12e804b76989e2a1a10b9361b P d645f2e569134a37baea2860781c5e0e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 01:19:27.096575 3635 tablet.cc:2369] T 15759fb12e804b76989e2a1a10b9361b P 06e0895a4bc34d5dad8a872a6fe475eb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 01:19:27.139007 4148 raft_consensus.cc:482] T 15759fb12e804b76989e2a1a10b9361b P d645f2e569134a37baea2860781c5e0e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 01:19:27.139196 4148 raft_consensus.cc:504] T 15759fb12e804b76989e2a1a10b9361b P d645f2e569134a37baea2860781c5e0e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06e0895a4bc34d5dad8a872a6fe475eb" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 34201 } } peers { permanent_uuid: "85a369874676488a8c43c1a2cf51a95a" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 41837 } } peers { permanent_uuid: "d645f2e569134a37baea2860781c5e0e" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 45783 } }
I20260812 01:19:27.139672 4148 leader_election.cc:288] T 15759fb12e804b76989e2a1a10b9361b P d645f2e569134a37baea2860781c5e0e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06e0895a4bc34d5dad8a872a6fe475eb (127.24.0.129:34201), 85a369874676488a8c43c1a2cf51a95a (127.24.0.131:41837)
I20260812 01:19:27.143416 3874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:27.139888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.131:41837 (local address 127.24.0.130:46651)
0812 01:19:27.139974 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:27.139984 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:27.140019 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:27.140193 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:27.140194 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:27.140195 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:19:27.140311 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:27.140313 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:27.141235 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:27.141236 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:27.142562 (+ 1326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:27.142566 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:27.143150 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:27.143151 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:27.143243 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:27.143247 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 01:19:27.143303 (+ 56us) client_negotiation.cc:240] Negotiation successful
0812 01:19:27.143338 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72,"mutex_wait_us":79}
I20260812 01:19:27.143442 4137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:27.139985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.130:46651 (local address 127.24.0.131:41837)
0812 01:19:27.140042 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:27.140044 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:27.140054 (+ 10us) server_negotiation.cc:407] Connection header received
0812 01:19:27.140098 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:27.140099 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:27.140111 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:27.140187 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:19:27.140394 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:27.141139 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:27.142674 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:27.143047 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:27.143143 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:27.143155 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:27.143318 (+ 163us) server_negotiation.cc:299] Negotiation successful
0812 01:19:27.143361 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 01:19:27.143540 3876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:27.139850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.129:34201 (local address 127.24.0.130:34353)
0812 01:19:27.139934 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:27.139946 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:27.139993 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:27.140193 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:27.140194 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:27.140195 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:19:27.140311 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:27.140313 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:27.141235 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:27.141236 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:27.142562 (+ 1326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:27.142566 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:27.143228 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:27.143229 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:27.143347 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:27.143350 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 01:19:27.143411 (+ 61us) client_negotiation.cc:240] Negotiation successful
0812 01:19:27.143456 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63,"mutex_wait_us":100}
I20260812 01:19:27.143561 4136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:27.139936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.130:34353 (local address 127.24.0.129:34201)
0812 01:19:27.139983 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:27.139985 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:27.139999 (+ 14us) server_negotiation.cc:407] Connection header received
0812 01:19:27.140092 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:27.140093 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:27.140105 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:27.140172 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:19:27.140395 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:27.141140 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:27.142690 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:27.143139 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:27.143216 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:27.143230 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:27.143433 (+ 203us) server_negotiation.cc:299] Negotiation successful
0812 01:19:27.143487 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 01:19:27.143905 3548 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15759fb12e804b76989e2a1a10b9361b" candidate_uuid: "d645f2e569134a37baea2860781c5e0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06e0895a4bc34d5dad8a872a6fe475eb" is_pre_election: true
I20260812 01:19:27.144097 3548 raft_consensus.cc:2454] T 15759fb12e804b76989e2a1a10b9361b P 06e0895a4bc34d5dad8a872a6fe475eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d645f2e569134a37baea2860781c5e0e in term 0.
I20260812 01:19:27.144106 4022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15759fb12e804b76989e2a1a10b9361b" candidate_uuid: "d645f2e569134a37baea2860781c5e0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85a369874676488a8c43c1a2cf51a95a" is_pre_election: true
I20260812 01:19:27.144268 4022 raft_consensus.cc:2454] T 15759fb12e804b76989e2a1a10b9361b P 85a369874676488a8c43c1a2cf51a95a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d645f2e569134a37baea2860781c5e0e in term 0.
I20260812 01:19:27.144383 3668 leader_election.cc:302] T 15759fb12e804b76989e2a1a10b9361b P d645f2e569134a37baea2860781c5e0e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06e0895a4bc34d5dad8a872a6fe475eb, d645f2e569134a37baea2860781c5e0e; no voters:
I20260812 01:19:27.144536 4148 raft_consensus.cc:2781] T 15759fb12e804b76989e2a1a10b9361b P d645f2e569134a37baea2860781c5e0e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 01:19:27.144603 4148 raft_consensus.cc:482] T 15759fb12e804b76989e2a1a10b9361b P d645f2e569134a37baea2860781c5e0e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 01:19:27.144630 4148 raft_consensus.cc:3037] T 15759fb12e804b76989e2a1a10b9361b P d645f2e569134a37baea2860781c5e0e [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:19:27.145300 4148 raft_consensus.cc:504] T 15759fb12e804b76989e2a1a10b9361b P d645f2e569134a37baea2860781c5e0e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06e0895a4bc34d5dad8a872a6fe475eb" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 34201 } } peers { permanent_uuid: "85a369874676488a8c43c1a2cf51a95a" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 41837 } } peers { permanent_uuid: "d645f2e569134a37baea2860781c5e0e" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 45783 } }
I20260812 01:19:27.145458 4148 leader_election.cc:288] T 15759fb12e804b76989e2a1a10b9361b P d645f2e569134a37baea2860781c5e0e [CANDIDATE]: Term 1 election: Requested vote from peers 06e0895a4bc34d5dad8a872a6fe475eb (127.24.0.129:34201), 85a369874676488a8c43c1a2cf51a95a (127.24.0.131:41837)
I20260812 01:19:27.145601 3548 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15759fb12e804b76989e2a1a10b9361b" candidate_uuid: "d645f2e569134a37baea2860781c5e0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06e0895a4bc34d5dad8a872a6fe475eb"
I20260812 01:19:27.145620 4022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15759fb12e804b76989e2a1a10b9361b" candidate_uuid: "d645f2e569134a37baea2860781c5e0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85a369874676488a8c43c1a2cf51a95a"
I20260812 01:19:27.145684 4022 raft_consensus.cc:3037] T 15759fb12e804b76989e2a1a10b9361b P 85a369874676488a8c43c1a2cf51a95a [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:19:27.145684 3548 raft_consensus.cc:3037] T 15759fb12e804b76989e2a1a10b9361b P 06e0895a4bc34d5dad8a872a6fe475eb [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:19:27.146571 4022 raft_consensus.cc:2454] T 15759fb12e804b76989e2a1a10b9361b P 85a369874676488a8c43c1a2cf51a95a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d645f2e569134a37baea2860781c5e0e in term 1.
I20260812 01:19:27.146580 3548 raft_consensus.cc:2454] T 15759fb12e804b76989e2a1a10b9361b P 06e0895a4bc34d5dad8a872a6fe475eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d645f2e569134a37baea2860781c5e0e in term 1.
I20260812 01:19:27.146757 3668 leader_election.cc:302] T 15759fb12e804b76989e2a1a10b9361b P d645f2e569134a37baea2860781c5e0e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06e0895a4bc34d5dad8a872a6fe475eb, d645f2e569134a37baea2860781c5e0e; no voters:
I20260812 01:19:27.146835 4148 raft_consensus.cc:2781] T 15759fb12e804b76989e2a1a10b9361b P d645f2e569134a37baea2860781c5e0e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:19:27.146982 4148 raft_consensus.cc:686] T 15759fb12e804b76989e2a1a10b9361b P d645f2e569134a37baea2860781c5e0e [term 1 LEADER]: Becoming Leader. State: Replica: d645f2e569134a37baea2860781c5e0e, State: Running, Role: LEADER
I20260812 01:19:27.147090 4148 consensus_queue.cc:237] T 15759fb12e804b76989e2a1a10b9361b P d645f2e569134a37baea2860781c5e0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06e0895a4bc34d5dad8a872a6fe475eb" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 34201 } } peers { permanent_uuid: "85a369874676488a8c43c1a2cf51a95a" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 41837 } } peers { permanent_uuid: "d645f2e569134a37baea2860781c5e0e" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 45783 } }
I20260812 01:19:27.147903 2035 catalog_manager.cc:5495] T 15759fb12e804b76989e2a1a10b9361b P d645f2e569134a37baea2860781c5e0e reported cstate change: term changed from 0 to 1, leader changed from <none> to d645f2e569134a37baea2860781c5e0e (127.24.0.130). New cstate: current_term: 1 leader_uuid: "d645f2e569134a37baea2860781c5e0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06e0895a4bc34d5dad8a872a6fe475eb" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 34201 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85a369874676488a8c43c1a2cf51a95a" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 41837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d645f2e569134a37baea2860781c5e0e" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 45783 } health_report { overall_health: HEALTHY } } }
W20260812 01:19:27.148168 4103 tablet.cc:2369] T 15759fb12e804b76989e2a1a10b9361b P 85a369874676488a8c43c1a2cf51a95a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 01:19:27.148890 4164 sys_catalog.cc:455] T 00000000000000000000000000000000 P efbfb4db2c524879a5d7d17801ff4cb0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "efbfb4db2c524879a5d7d17801ff4cb0" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "efbfb4db2c524879a5d7d17801ff4cb0" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 46409 } } peers { permanent_uuid: "35a50d99aa1b4d9c8a4b11849067ad0f" member_type: VOTER last_known_addr { host: "127.24.0.189" port: 38795 } attrs { promote: false } } peers { permanent_uuid: "ccb9bfbd91334a59bb9b562bc5be7fa6" member_type: NON_VOTER last_known_addr { host: "127.24.0.188" port: 37741 } attrs { promote: true } } }
I20260812 01:19:27.149010 4164 sys_catalog.cc:458] T 00000000000000000000000000000000 P efbfb4db2c524879a5d7d17801ff4cb0 [sys.catalog]: This master's current role is: LEADER
I20260812 01:19:27.149590 3909 raft_consensus.cc:1050] T 00000000000000000000000000000000 P efbfb4db2c524879a5d7d17801ff4cb0: attempting to promote NON_VOTER ccb9bfbd91334a59bb9b562bc5be7fa6 to VOTER
I20260812 01:19:27.149756 3909 consensus_queue.cc:237] T 00000000000000000000000000000000 P efbfb4db2c524879a5d7d17801ff4cb0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "efbfb4db2c524879a5d7d17801ff4cb0" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 46409 } } peers { permanent_uuid: "35a50d99aa1b4d9c8a4b11849067ad0f" member_type: VOTER last_known_addr { host: "127.24.0.189" port: 38795 } attrs { promote: false } } peers { permanent_uuid: "ccb9bfbd91334a59bb9b562bc5be7fa6" member_type: VOTER last_known_addr { host: "127.24.0.188" port: 37741 } attrs { promote: false } }
I20260812 01:19:27.150241 3338 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ccb9bfbd91334a59bb9b562bc5be7fa6 [term 1 LEARNER]: Refusing update from remote peer efbfb4db2c524879a5d7d17801ff4cb0: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260812 01:19:27.150264 3049 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 35a50d99aa1b4d9c8a4b11849067ad0f [term 1 FOLLOWER]: Refusing update from remote peer efbfb4db2c524879a5d7d17801ff4cb0: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260812 01:19:27.150454 4164 consensus_queue.cc:1035] T 00000000000000000000000000000000 P efbfb4db2c524879a5d7d17801ff4cb0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccb9bfbd91334a59bb9b562bc5be7fa6" member_type: VOTER last_known_addr { host: "127.24.0.188" port: 37741 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260812 01:19:27.150544 4124 consensus_queue.cc:1035] T 00000000000000000000000000000000 P efbfb4db2c524879a5d7d17801ff4cb0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35a50d99aa1b4d9c8a4b11849067ad0f" member_type: VOTER last_known_addr { host: "127.24.0.189" port: 38795 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260812 01:19:27.151245 3909 raft_consensus.cc:2932] T 00000000000000000000000000000000 P efbfb4db2c524879a5d7d17801ff4cb0 [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 7 to 11, ccb9bfbd91334a59bb9b562bc5be7fa6 (127.24.0.188) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "efbfb4db2c524879a5d7d17801ff4cb0" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 46409 } } peers { permanent_uuid: "35a50d99aa1b4d9c8a4b11849067ad0f" member_type: VOTER last_known_addr { host: "127.24.0.189" port: 38795 } attrs { promote: false } } peers { permanent_uuid: "ccb9bfbd91334a59bb9b562bc5be7fa6" member_type: VOTER last_known_addr { host: "127.24.0.188" port: 37741 } attrs { promote: false } } }
I20260812 01:19:27.151350 3049 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 35a50d99aa1b4d9c8a4b11849067ad0f [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, ccb9bfbd91334a59bb9b562bc5be7fa6 (127.24.0.188) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "efbfb4db2c524879a5d7d17801ff4cb0" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 46409 } } peers { permanent_uuid: "35a50d99aa1b4d9c8a4b11849067ad0f" member_type: VOTER last_known_addr { host: "127.24.0.189" port: 38795 } attrs { promote: false } } peers { permanent_uuid: "ccb9bfbd91334a59bb9b562bc5be7fa6" member_type: VOTER last_known_addr { host: "127.24.0.188" port: 37741 } attrs { promote: false } } }
I20260812 01:19:27.151386 3338 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ccb9bfbd91334a59bb9b562bc5be7fa6 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, ccb9bfbd91334a59bb9b562bc5be7fa6 (127.24.0.188) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "efbfb4db2c524879a5d7d17801ff4cb0" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 46409 } } peers { permanent_uuid: "35a50d99aa1b4d9c8a4b11849067ad0f" member_type: VOTER last_known_addr { host: "127.24.0.189" port: 38795 } attrs { promote: false } } peers { permanent_uuid: "ccb9bfbd91334a59bb9b562bc5be7fa6" member_type: VOTER last_known_addr { host: "127.24.0.188" port: 37741 } attrs { promote: false } } }
I20260812 01:19:27.152220 3909 sys_catalog.cc:455] T 00000000000000000000000000000000 P efbfb4db2c524879a5d7d17801ff4cb0 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "efbfb4db2c524879a5d7d17801ff4cb0" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "efbfb4db2c524879a5d7d17801ff4cb0" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 46409 } } peers { permanent_uuid: "35a50d99aa1b4d9c8a4b11849067ad0f" member_type: VOTER last_known_addr { host: "127.24.0.189" port: 38795 } attrs { promote: false } } peers { permanent_uuid: "ccb9bfbd91334a59bb9b562bc5be7fa6" member_type: VOTER last_known_addr { host: "127.24.0.188" port: 37741 } attrs { promote: false } } }
I20260812 01:19:27.152318 3909 sys_catalog.cc:458] T 00000000000000000000000000000000 P efbfb4db2c524879a5d7d17801ff4cb0 [sys.catalog]: This master's current role is: LEADER
I20260812 01:19:27.152383 3909 sys_catalog.cc:455] T 00000000000000000000000000000000 P efbfb4db2c524879a5d7d17801ff4cb0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "efbfb4db2c524879a5d7d17801ff4cb0" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "efbfb4db2c524879a5d7d17801ff4cb0" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 46409 } } peers { permanent_uuid: "35a50d99aa1b4d9c8a4b11849067ad0f" member_type: VOTER last_known_addr { host: "127.24.0.189" port: 38795 } attrs { promote: false } } peers { permanent_uuid: "ccb9bfbd91334a59bb9b562bc5be7fa6" member_type: VOTER last_known_addr { host: "127.24.0.188" port: 37741 } attrs { promote: false } } }
I20260812 01:19:27.152449 3909 sys_catalog.cc:458] T 00000000000000000000000000000000 P efbfb4db2c524879a5d7d17801ff4cb0 [sys.catalog]: This master's current role is: LEADER
I20260812 01:19:27.152423 4153 sys_catalog.cc:455] T 00000000000000000000000000000000 P ccb9bfbd91334a59bb9b562bc5be7fa6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "efbfb4db2c524879a5d7d17801ff4cb0" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "efbfb4db2c524879a5d7d17801ff4cb0" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 46409 } } peers { permanent_uuid: "35a50d99aa1b4d9c8a4b11849067ad0f" member_type: VOTER last_known_addr { host: "127.24.0.189" port: 38795 } attrs { promote: false } } peers { permanent_uuid: "ccb9bfbd91334a59bb9b562bc5be7fa6" member_type: VOTER last_known_addr { host: "127.24.0.188" port: 37741 } attrs { promote: false } } }
I20260812 01:19:27.152508 4153 sys_catalog.cc:458] T 00000000000000000000000000000000 P ccb9bfbd91334a59bb9b562bc5be7fa6 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 01:19:27.152618 4164 sys_catalog.cc:455] T 00000000000000000000000000000000 P efbfb4db2c524879a5d7d17801ff4cb0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "efbfb4db2c524879a5d7d17801ff4cb0" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "efbfb4db2c524879a5d7d17801ff4cb0" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 46409 } } peers { permanent_uuid: "35a50d99aa1b4d9c8a4b11849067ad0f" member_type: VOTER last_known_addr { host: "127.24.0.189" port: 38795 } attrs { promote: false } } peers { permanent_uuid: "ccb9bfbd91334a59bb9b562bc5be7fa6" member_type: VOTER last_known_addr { host: "127.24.0.188" port: 37741 } attrs { promote: false } } }
I20260812 01:19:27.152719 4164 sys_catalog.cc:458] T 00000000000000000000000000000000 P efbfb4db2c524879a5d7d17801ff4cb0 [sys.catalog]: This master's current role is: LEADER
I20260812 01:19:27.152736 4173 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35a50d99aa1b4d9c8a4b11849067ad0f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "efbfb4db2c524879a5d7d17801ff4cb0" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "efbfb4db2c524879a5d7d17801ff4cb0" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 46409 } } peers { permanent_uuid: "35a50d99aa1b4d9c8a4b11849067ad0f" member_type: VOTER last_known_addr { host: "127.24.0.189" port: 38795 } attrs { promote: false } } peers { permanent_uuid: "ccb9bfbd91334a59bb9b562bc5be7fa6" member_type: VOTER last_known_addr { host: "127.24.0.188" port: 37741 } attrs { promote: false } } }
I20260812 01:19:27.152822 4173 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35a50d99aa1b4d9c8a4b11849067ad0f [sys.catalog]: This master's current role is: FOLLOWER
I20260812 01:19:27.210750 4138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:27.208397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32898 (local address 127.24.0.130:45783)
0812 01:19:27.208452 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:27.208454 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:27.208466 (+ 12us) server_negotiation.cc:407] Connection header received
0812 01:19:27.208514 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:27.208515 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:27.208537 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:27.208608 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:19:27.208719 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:27.209264 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:27.209959 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:27.210205 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:27.210293 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:27.210390 (+ 97us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:19:27.210505 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:19:27.210569 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:27.210600 (+ 31us) server_negotiation.cc:299] Negotiation successful
0812 01:19:27.210620 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260812 01:19:27.212905 3548 raft_consensus.cc:1261] T 15759fb12e804b76989e2a1a10b9361b P 06e0895a4bc34d5dad8a872a6fe475eb [term 1 FOLLOWER]: Refusing update from remote peer d645f2e569134a37baea2860781c5e0e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:19:27.212915 4022 raft_consensus.cc:1261] T 15759fb12e804b76989e2a1a10b9361b P 85a369874676488a8c43c1a2cf51a95a [term 1 FOLLOWER]: Refusing update from remote peer d645f2e569134a37baea2860781c5e0e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:19:27.213306 4148 consensus_queue.cc:1035] T 15759fb12e804b76989e2a1a10b9361b P d645f2e569134a37baea2860781c5e0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "85a369874676488a8c43c1a2cf51a95a" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 41837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:19:27.213474 4160 consensus_queue.cc:1035] T 15759fb12e804b76989e2a1a10b9361b P d645f2e569134a37baea2860781c5e0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "06e0895a4bc34d5dad8a872a6fe475eb" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 34201 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:19:27.216915 4181 mvcc.cc:205] Tried to move back new op lower bound from 7317494035300917248 to 7317494035035123712. Current Snapshot: MvccSnapshot[applied={T|T < 7317494035300917248}]
I20260812 01:19:27.217504 4178 mvcc.cc:205] Tried to move back new op lower bound from 7317494035300917248 to 7317494035035123712. Current Snapshot: MvccSnapshot[applied={T|T < 7317494035300917248}]
I20260812 01:19:27.220260 2033 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60114:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:19:27.220350 2033 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60114:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:19:27.221802 2033 catalog_manager.cc:5782] T 00000000000000000000000000000000 P efbfb4db2c524879a5d7d17801ff4cb0: Sending DeleteTablet for 3 replicas of tablet 15759fb12e804b76989e2a1a10b9361b
I20260812 01:19:27.222164 3516 tablet_service.cc:1514] Processing DeleteTablet for tablet 15759fb12e804b76989e2a1a10b9361b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:19:27 UTC) from {username='oryx', principal='oryx/127.24.0.190@KRBTEST.COM'} at 127.0.0.1:33812
I20260812 01:19:27.222276 3989 tablet_service.cc:1514] Processing DeleteTablet for tablet 15759fb12e804b76989e2a1a10b9361b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:19:27 UTC) from {username='oryx', principal='oryx/127.24.0.190@KRBTEST.COM'} at 127.0.0.1:59774
I20260812 01:19:27.222484 4193 tablet_replica.cc:330] T 15759fb12e804b76989e2a1a10b9361b P 06e0895a4bc34d5dad8a872a6fe475eb: stopping tablet replica
I20260812 01:19:27.222501 4194 tablet_replica.cc:330] T 15759fb12e804b76989e2a1a10b9361b P 85a369874676488a8c43c1a2cf51a95a: stopping tablet replica
I20260812 01:19:27.222587 4194 raft_consensus.cc:2229] T 15759fb12e804b76989e2a1a10b9361b P 85a369874676488a8c43c1a2cf51a95a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 01:19:27.222587 4193 raft_consensus.cc:2229] T 15759fb12e804b76989e2a1a10b9361b P 06e0895a4bc34d5dad8a872a6fe475eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 01:19:27.222687 4194 raft_consensus.cc:2258] T 15759fb12e804b76989e2a1a10b9361b P 85a369874676488a8c43c1a2cf51a95a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:19:27.222698 4193 raft_consensus.cc:2258] T 15759fb12e804b76989e2a1a10b9361b P 06e0895a4bc34d5dad8a872a6fe475eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:19:27.222831 3754 tablet_service.cc:1514] Processing DeleteTablet for tablet 15759fb12e804b76989e2a1a10b9361b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:19:27 UTC) from {username='oryx', principal='oryx/127.24.0.190@KRBTEST.COM'} at 127.0.0.1:32896
I20260812 01:19:27.223147 4197 tablet_replica.cc:330] T 15759fb12e804b76989e2a1a10b9361b P d645f2e569134a37baea2860781c5e0e: stopping tablet replica
I20260812 01:19:27.223259 4197 raft_consensus.cc:2229] T 15759fb12e804b76989e2a1a10b9361b P d645f2e569134a37baea2860781c5e0e [term 1 LEADER]: Raft consensus shutting down.
I20260812 01:19:27.223340 4194 ts_tablet_manager.cc:1905] T 15759fb12e804b76989e2a1a10b9361b P 85a369874676488a8c43c1a2cf51a95a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:19:27.223369 4197 raft_consensus.cc:2258] T 15759fb12e804b76989e2a1a10b9361b P d645f2e569134a37baea2860781c5e0e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:19:27.224400 4197 ts_tablet_manager.cc:1905] T 15759fb12e804b76989e2a1a10b9361b P d645f2e569134a37baea2860781c5e0e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:19:27.224455 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 1984
I20260812 01:19:27.227093 4197 ts_tablet_manager.cc:1918] T 15759fb12e804b76989e2a1a10b9361b P d645f2e569134a37baea2860781c5e0e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 01:19:27.227093 4194 ts_tablet_manager.cc:1918] T 15759fb12e804b76989e2a1a10b9361b P 85a369874676488a8c43c1a2cf51a95a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 01:19:27.227164 4194 log.cc:1196] T 15759fb12e804b76989e2a1a10b9361b P 85a369874676488a8c43c1a2cf51a95a: Deleting WAL directory at /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/ts-2/wal/wals/15759fb12e804b76989e2a1a10b9361b
I20260812 01:19:27.227164 4197 log.cc:1196] T 15759fb12e804b76989e2a1a10b9361b P d645f2e569134a37baea2860781c5e0e: Deleting WAL directory at /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/ts-1/wal/wals/15759fb12e804b76989e2a1a10b9361b
I20260812 01:19:27.227451 4194 ts_tablet_manager.cc:1939] T 15759fb12e804b76989e2a1a10b9361b P 85a369874676488a8c43c1a2cf51a95a: Deleting consensus metadata
I20260812 01:19:27.227451 4197 ts_tablet_manager.cc:1939] T 15759fb12e804b76989e2a1a10b9361b P d645f2e569134a37baea2860781c5e0e: Deleting consensus metadata
I20260812 01:19:27.236510 4193 ts_tablet_manager.cc:1905] T 15759fb12e804b76989e2a1a10b9361b P 06e0895a4bc34d5dad8a872a6fe475eb: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260812 01:19:27.237619 3675 connection.cc:535] server connection from 127.0.0.1:32896 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260812 01:19:27.237911 3916 connection.cc:535] server connection from 127.0.0.1:59774 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 01:19:27.238204 4193 ts_tablet_manager.cc:1918] T 15759fb12e804b76989e2a1a10b9361b P 06e0895a4bc34d5dad8a872a6fe475eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 01:19:27.238287 4193 log.cc:1196] T 15759fb12e804b76989e2a1a10b9361b P 06e0895a4bc34d5dad8a872a6fe475eb: Deleting WAL directory at /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/ts-0/wal/wals/15759fb12e804b76989e2a1a10b9361b
I20260812 01:19:27.238571 4193 ts_tablet_manager.cc:1939] T 15759fb12e804b76989e2a1a10b9361b P 06e0895a4bc34d5dad8a872a6fe475eb: Deleting consensus metadata
W20260812 01:19:27.238963 3435 connection.cc:441] server connection from 127.0.0.1:33812 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33812 (request call id 1) could send its response
I20260812 01:19:27.247751 4116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:27.239682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54998 (local address 127.24.0.188:37741)
0812 01:19:27.239737 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:27.239739 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:27.239888 (+ 149us) server_negotiation.cc:407] Connection header received
0812 01:19:27.239989 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:27.239991 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:27.240002 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:27.240052 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:27.240720 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:27.241492 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:27.242602 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:27.242902 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:27.242967 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:27.244431 (+ 1464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:27.244432 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:27.244433 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:27.244440 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:27.246497 (+ 2057us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:27.246938 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:27.246940 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:27.246940 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:27.247003 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:27.247217 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:27.247218 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:27.247218 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:27.247354 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:27.247453 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:27.247566 (+ 113us) server_negotiation.cc:299] Negotiation successful
0812 01:19:27.247656 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 01:19:27.247782 4122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:27.239844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44146 (local address 127.24.0.189:38795)
0812 01:19:27.239892 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:27.239894 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:27.239983 (+ 89us) server_negotiation.cc:407] Connection header received
0812 01:19:27.240083 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:27.240084 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:27.240098 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:27.240144 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:27.240863 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:27.241583 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:27.242457 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:27.242771 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:27.242842 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:27.244487 (+ 1645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:27.244489 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:27.244489 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:27.244496 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:27.246377 (+ 1881us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:27.246830 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:27.246831 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:27.246832 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:27.246894 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:27.247135 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:27.247136 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:27.247136 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:27.247274 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:27.247338 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:27.247571 (+ 233us) server_negotiation.cc:299] Negotiation successful
0812 01:19:27.247653 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 01:19:28.218704 4218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:28.218145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:46409 (local address 127.24.0.130:51917)
0812 01:19:28.218470 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:28.218623 (+ 153us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:46409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":69,"threads_started":1}
I20260812 01:19:28.218789 4223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:28.218311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:46409 (local address 127.24.0.129:59111)
0812 01:19:28.218669 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:28.218741 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:46409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":54,"threads_started":1}
W20260812 01:19:28.218920 3868 heartbeater.cc:644] Failed to heartbeat to 127.24.0.190:46409 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.24.0.190:46409: connect: Connection refused (error 111)
W20260812 01:19:28.219070 3630 heartbeater.cc:644] Failed to heartbeat to 127.24.0.190:46409 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.24.0.190:46409: connect: Connection refused (error 111)
I20260812 01:19:28.219097 4212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:28.218110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:46409 (local address 127.24.0.131:45201)
0812 01:19:28.218376 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:28.219050 (+ 674us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:46409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":82,"threads_started":1}
W20260812 01:19:28.219338 4101 heartbeater.cc:644] Failed to heartbeat to 127.24.0.190:46409 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.24.0.190:46409: connect: Connection refused (error 111)
I20260812 01:19:28.722456 4226 raft_consensus.cc:482] T 00000000000000000000000000000000 P 35a50d99aa1b4d9c8a4b11849067ad0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader efbfb4db2c524879a5d7d17801ff4cb0)
I20260812 01:19:28.722597 4226 raft_consensus.cc:504] T 00000000000000000000000000000000 P 35a50d99aa1b4d9c8a4b11849067ad0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "efbfb4db2c524879a5d7d17801ff4cb0" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 46409 } } peers { permanent_uuid: "35a50d99aa1b4d9c8a4b11849067ad0f" member_type: VOTER last_known_addr { host: "127.24.0.189" port: 38795 } attrs { promote: false } } peers { permanent_uuid: "ccb9bfbd91334a59bb9b562bc5be7fa6" member_type: VOTER last_known_addr { host: "127.24.0.188" port: 37741 } attrs { promote: false } }
I20260812 01:19:28.722826 4228 raft_consensus.cc:482] T 00000000000000000000000000000000 P ccb9bfbd91334a59bb9b562bc5be7fa6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader efbfb4db2c524879a5d7d17801ff4cb0)
I20260812 01:19:28.722959 4226 leader_election.cc:288] T 00000000000000000000000000000000 P 35a50d99aa1b4d9c8a4b11849067ad0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers efbfb4db2c524879a5d7d17801ff4cb0 (127.24.0.190:46409), ccb9bfbd91334a59bb9b562bc5be7fa6 (127.24.0.188:37741)
I20260812 01:19:28.722976 4228 raft_consensus.cc:504] T 00000000000000000000000000000000 P ccb9bfbd91334a59bb9b562bc5be7fa6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "efbfb4db2c524879a5d7d17801ff4cb0" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 46409 } } peers { permanent_uuid: "35a50d99aa1b4d9c8a4b11849067ad0f" member_type: VOTER last_known_addr { host: "127.24.0.189" port: 38795 } attrs { promote: false } } peers { permanent_uuid: "ccb9bfbd91334a59bb9b562bc5be7fa6" member_type: VOTER last_known_addr { host: "127.24.0.188" port: 37741 } attrs { promote: false } }
I20260812 01:19:28.723307 4229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:28.723087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:46409 (local address 127.0.0.1:60554)
0812 01:19:28.723206 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:28.723258 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:46409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":40,"threads_started":1}
W20260812 01:19:28.723424 2996 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.0.190:46409: connect: Connection refused (error 111)
I20260812 01:19:28.723562 4231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:28.723320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:46409 (local address 127.0.0.1:60560)
0812 01:19:28.723464 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:28.723511 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:46409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":38,"threads_started":1}
W20260812 01:19:28.723807 3306 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.0.190:46409: connect: Connection refused (error 111)
I20260812 01:19:28.723919 4229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:28.723822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:46409 (local address 127.0.0.1:60574)
0812 01:19:28.723860 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:28.723886 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:46409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260812 01:19:28.724004 2996 leader_election.cc:334] T 00000000000000000000000000000000 P 35a50d99aa1b4d9c8a4b11849067ad0f [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer efbfb4db2c524879a5d7d17801ff4cb0 (127.24.0.190:46409): Network error: Client connection negotiation failed: client connection to 127.24.0.190:46409: connect: Connection refused (error 111)
I20260812 01:19:28.724045 4228 leader_election.cc:288] T 00000000000000000000000000000000 P ccb9bfbd91334a59bb9b562bc5be7fa6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers efbfb4db2c524879a5d7d17801ff4cb0 (127.24.0.190:46409), 35a50d99aa1b4d9c8a4b11849067ad0f (127.24.0.189:38795)
I20260812 01:19:28.724505 4234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:28.724318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:46409 (local address 127.0.0.1:60580)
0812 01:19:28.724381 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:28.724417 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:46409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260812 01:19:28.724576 3306 leader_election.cc:334] T 00000000000000000000000000000000 P ccb9bfbd91334a59bb9b562bc5be7fa6 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer efbfb4db2c524879a5d7d17801ff4cb0 (127.24.0.190:46409): Network error: Client connection negotiation failed: client connection to 127.24.0.190:46409: connect: Connection refused (error 111)
I20260812 01:19:28.726815 4230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:28.723176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.188:37741 (local address 127.0.0.1:55002)
0812 01:19:28.723469 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:28.723522 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:28.723718 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:28.724233 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:28.724234 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:28.724237 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:19:28.724293 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:28.724295 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:28.725222 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:28.725223 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:28.725961 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:28.725963 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:28.726537 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:28.726538 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:28.726609 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:28.726612 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 01:19:28.726691 (+ 79us) client_negotiation.cc:240] Negotiation successful
0812 01:19:28.726738 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":54,"threads_started":1}
I20260812 01:19:28.726801 4232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:28.723315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55002 (local address 127.24.0.188:37741)
0812 01:19:28.723416 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:28.723418 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:28.723584 (+ 166us) server_negotiation.cc:407] Connection header received
0812 01:19:28.723816 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:28.723817 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:28.723830 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:28.723918 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:19:28.724378 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:28.725119 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:28.726056 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:28.726450 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:28.726538 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:28.726551 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:28.726700 (+ 149us) server_negotiation.cc:299] Negotiation successful
0812 01:19:28.726743 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81,"thread_start_us":54,"threads_started":1}
I20260812 01:19:28.727011 4236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:28.723648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44160 (local address 127.24.0.189:38795)
0812 01:19:28.723822 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:28.723825 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:28.723832 (+ 7us) server_negotiation.cc:407] Connection header received
0812 01:19:28.723855 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:28.723856 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:28.723866 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:28.724196 (+ 330us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:19:28.724238 (+ 42us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:28.725103 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:28.726110 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:28.726498 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:28.726594 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:28.726607 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:28.726760 (+ 153us) server_negotiation.cc:299] Negotiation successful
0812 01:19:28.726804 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":83,"threads_started":1}
I20260812 01:19:28.727087 3338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "35a50d99aa1b4d9c8a4b11849067ad0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "ccb9bfbd91334a59bb9b562bc5be7fa6" is_pre_election: true
I20260812 01:19:28.727195 3338 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ccb9bfbd91334a59bb9b562bc5be7fa6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35a50d99aa1b4d9c8a4b11849067ad0f in term 1.
I20260812 01:19:28.727278 4231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:28.723505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.189:38795 (local address 127.0.0.1:44160)
0812 01:19:28.723635 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:28.723644 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:28.723715 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:28.723962 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:28.723963 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:28.723966 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:19:28.724027 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:28.724029 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:28.725214 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:28.725215 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:28.725995 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:28.725997 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:28.726600 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:28.726601 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:28.726669 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:28.726674 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 01:19:28.726751 (+ 77us) client_negotiation.cc:240] Negotiation successful
0812 01:19:28.726796 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":66,"threads_started":1}
I20260812 01:19:28.727365 2999 leader_election.cc:302] T 00000000000000000000000000000000 P 35a50d99aa1b4d9c8a4b11849067ad0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 35a50d99aa1b4d9c8a4b11849067ad0f, ccb9bfbd91334a59bb9b562bc5be7fa6; no voters: efbfb4db2c524879a5d7d17801ff4cb0
I20260812 01:19:28.727481 4226 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 35a50d99aa1b4d9c8a4b11849067ad0f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260812 01:19:28.727541 4226 raft_consensus.cc:482] T 00000000000000000000000000000000 P 35a50d99aa1b4d9c8a4b11849067ad0f [term 1 FOLLOWER]: Starting leader election (detected failure of leader efbfb4db2c524879a5d7d17801ff4cb0)
I20260812 01:19:28.727563 4226 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 35a50d99aa1b4d9c8a4b11849067ad0f [term 1 FOLLOWER]: Advancing to term 2
I20260812 01:19:28.727552 3049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ccb9bfbd91334a59bb9b562bc5be7fa6" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "35a50d99aa1b4d9c8a4b11849067ad0f" is_pre_election: true
I20260812 01:19:28.728583 4226 raft_consensus.cc:504] T 00000000000000000000000000000000 P 35a50d99aa1b4d9c8a4b11849067ad0f [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "efbfb4db2c524879a5d7d17801ff4cb0" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 46409 } } peers { permanent_uuid: "35a50d99aa1b4d9c8a4b11849067ad0f" member_type: VOTER last_known_addr { host: "127.24.0.189" port: 38795 } attrs { promote: false } } peers { permanent_uuid: "ccb9bfbd91334a59bb9b562bc5be7fa6" member_type: VOTER last_known_addr { host: "127.24.0.188" port: 37741 } attrs { promote: false } }
I20260812 01:19:28.728756 4226 leader_election.cc:288] T 00000000000000000000000000000000 P 35a50d99aa1b4d9c8a4b11849067ad0f [CANDIDATE]: Term 2 election: Requested vote from peers efbfb4db2c524879a5d7d17801ff4cb0 (127.24.0.190:46409), ccb9bfbd91334a59bb9b562bc5be7fa6 (127.24.0.188:37741)
I20260812 01:19:28.728763 3049 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 35a50d99aa1b4d9c8a4b11849067ad0f [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ccb9bfbd91334a59bb9b562bc5be7fa6 in current term 2: Already voted for candidate 35a50d99aa1b4d9c8a4b11849067ad0f in this term.
I20260812 01:19:28.729035 4230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:28.728899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:46409 (local address 127.0.0.1:60584)
0812 01:19:28.728948 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:28.728973 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:46409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260812 01:19:28.729434 4230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:28.729348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:46409 (local address 127.0.0.1:60594)
0812 01:19:28.729377 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:28.729414 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:46409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":13}
W20260812 01:19:28.729512 2996 leader_election.cc:334] T 00000000000000000000000000000000 P 35a50d99aa1b4d9c8a4b11849067ad0f [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer efbfb4db2c524879a5d7d17801ff4cb0 (127.24.0.190:46409): Network error: Client connection negotiation failed: client connection to 127.24.0.190:46409: connect: Connection refused (error 111)
I20260812 01:19:28.736886 3306 leader_election.cc:302] T 00000000000000000000000000000000 P ccb9bfbd91334a59bb9b562bc5be7fa6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccb9bfbd91334a59bb9b562bc5be7fa6; no voters: 35a50d99aa1b4d9c8a4b11849067ad0f, efbfb4db2c524879a5d7d17801ff4cb0
I20260812 01:19:28.736929 3338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "35a50d99aa1b4d9c8a4b11849067ad0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "ccb9bfbd91334a59bb9b562bc5be7fa6"
I20260812 01:19:28.737022 3338 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ccb9bfbd91334a59bb9b562bc5be7fa6 [term 1 FOLLOWER]: Advancing to term 2
I20260812 01:19:28.737975 3338 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ccb9bfbd91334a59bb9b562bc5be7fa6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35a50d99aa1b4d9c8a4b11849067ad0f in term 2.
I20260812 01:19:28.738029 4228 raft_consensus.cc:2726] T 00000000000000000000000000000000 P ccb9bfbd91334a59bb9b562bc5be7fa6 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 01:19:28.738157 2999 leader_election.cc:302] T 00000000000000000000000000000000 P 35a50d99aa1b4d9c8a4b11849067ad0f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 35a50d99aa1b4d9c8a4b11849067ad0f, ccb9bfbd91334a59bb9b562bc5be7fa6; no voters: efbfb4db2c524879a5d7d17801ff4cb0
I20260812 01:19:28.738265 4226 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 35a50d99aa1b4d9c8a4b11849067ad0f [term 2 FOLLOWER]: Leader election won for term 2
I20260812 01:19:28.738448 4226 raft_consensus.cc:686] T 00000000000000000000000000000000 P 35a50d99aa1b4d9c8a4b11849067ad0f [term 2 LEADER]: Becoming Leader. State: Replica: 35a50d99aa1b4d9c8a4b11849067ad0f, State: Running, Role: LEADER
I20260812 01:19:28.738514 4226 consensus_queue.cc:237] T 00000000000000000000000000000000 P 35a50d99aa1b4d9c8a4b11849067ad0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "efbfb4db2c524879a5d7d17801ff4cb0" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 46409 } } peers { permanent_uuid: "35a50d99aa1b4d9c8a4b11849067ad0f" member_type: VOTER last_known_addr { host: "127.24.0.189" port: 38795 } attrs { promote: false } } peers { permanent_uuid: "ccb9bfbd91334a59bb9b562bc5be7fa6" member_type: VOTER last_known_addr { host: "127.24.0.188" port: 37741 } attrs { promote: false } }
I20260812 01:19:28.738878 4238 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35a50d99aa1b4d9c8a4b11849067ad0f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 35a50d99aa1b4d9c8a4b11849067ad0f. Latest consensus state: current_term: 2 leader_uuid: "35a50d99aa1b4d9c8a4b11849067ad0f" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "efbfb4db2c524879a5d7d17801ff4cb0" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 46409 } } peers { permanent_uuid: "35a50d99aa1b4d9c8a4b11849067ad0f" member_type: VOTER last_known_addr { host: "127.24.0.189" port: 38795 } attrs { promote: false } } peers { permanent_uuid: "ccb9bfbd91334a59bb9b562bc5be7fa6" member_type: VOTER last_known_addr { host: "127.24.0.188" port: 37741 } attrs { promote: false } } }
I20260812 01:19:28.738958 4238 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35a50d99aa1b4d9c8a4b11849067ad0f [sys.catalog]: This master's current role is: LEADER
I20260812 01:19:28.739140 4241 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:19:28.739557 4241 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:19:28.739610 4241 catalog_manager.cc:1259] Loaded cluster ID: 702944a007944b5db36839527bbbf20f
I20260812 01:19:28.739657 4241 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 01:19:28.739848 4241 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:19:28.739948 4241 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 35a50d99aa1b4d9c8a4b11849067ad0f: Loaded TSK: 0
I20260812 01:19:28.740154 4241 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 01:19:28.785912 3031 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44146:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 01:19:28.787685 3031 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 01:19:28.788836 4230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:28.788699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:46409 (local address 127.0.0.1:60612)
0812 01:19:28.788752 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:28.788789 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:46409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260812 01:19:28.788867 3338 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ccb9bfbd91334a59bb9b562bc5be7fa6 [term 2 FOLLOWER]: Refusing update from remote peer 35a50d99aa1b4d9c8a4b11849067ad0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260812 01:19:28.789193 4230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:28.789093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:46409 (local address 127.0.0.1:60628)
0812 01:19:28.789142 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:28.789174 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:46409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
I20260812 01:19:28.789155 4226 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 35a50d99aa1b4d9c8a4b11849067ad0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccb9bfbd91334a59bb9b562bc5be7fa6" member_type: VOTER last_known_addr { host: "127.24.0.188" port: 37741 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260812 01:19:28.789280 2996 consensus_peers.cc:487] T 00000000000000000000000000000000 P 35a50d99aa1b4d9c8a4b11849067ad0f -> Peer efbfb4db2c524879a5d7d17801ff4cb0 (127.24.0.190:46409): Couldn't send request to peer efbfb4db2c524879a5d7d17801ff4cb0. Status: Network error: Client connection negotiation failed: client connection to 127.24.0.190:46409: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260812 01:19:28.789820 4228 sys_catalog.cc:455] T 00000000000000000000000000000000 P ccb9bfbd91334a59bb9b562bc5be7fa6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 35a50d99aa1b4d9c8a4b11849067ad0f. Latest consensus state: current_term: 2 leader_uuid: "35a50d99aa1b4d9c8a4b11849067ad0f" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "efbfb4db2c524879a5d7d17801ff4cb0" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 46409 } } peers { permanent_uuid: "35a50d99aa1b4d9c8a4b11849067ad0f" member_type: VOTER last_known_addr { host: "127.24.0.189" port: 38795 } attrs { promote: false } } peers { permanent_uuid: "ccb9bfbd91334a59bb9b562bc5be7fa6" member_type: VOTER last_known_addr { host: "127.24.0.188" port: 37741 } attrs { promote: false } } }
I20260812 01:19:28.789906 4228 sys_catalog.cc:458] T 00000000000000000000000000000000 P ccb9bfbd91334a59bb9b562bc5be7fa6 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 01:19:28.790277 4242 mvcc.cc:205] Tried to move back new op lower bound from 7317494041756844032 to 7317494041553625088. Current Snapshot: MvccSnapshot[applied={T|T < 7317494041756844032}]
I20260812 01:19:28.790392 4226 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35a50d99aa1b4d9c8a4b11849067ad0f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "35a50d99aa1b4d9c8a4b11849067ad0f" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "efbfb4db2c524879a5d7d17801ff4cb0" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 46409 } } peers { permanent_uuid: "35a50d99aa1b4d9c8a4b11849067ad0f" member_type: VOTER last_known_addr { host: "127.24.0.189" port: 38795 } attrs { promote: false } } peers { permanent_uuid: "ccb9bfbd91334a59bb9b562bc5be7fa6" member_type: VOTER last_known_addr { host: "127.24.0.188" port: 37741 } attrs { promote: false } } }
I20260812 01:19:28.790464 4226 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35a50d99aa1b4d9c8a4b11849067ad0f [sys.catalog]: This master's current role is: LEADER
I20260812 01:19:28.790467 4228 sys_catalog.cc:455] T 00000000000000000000000000000000 P ccb9bfbd91334a59bb9b562bc5be7fa6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "35a50d99aa1b4d9c8a4b11849067ad0f" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "efbfb4db2c524879a5d7d17801ff4cb0" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 46409 } } peers { permanent_uuid: "35a50d99aa1b4d9c8a4b11849067ad0f" member_type: VOTER last_known_addr { host: "127.24.0.189" port: 38795 } attrs { promote: false } } peers { permanent_uuid: "ccb9bfbd91334a59bb9b562bc5be7fa6" member_type: VOTER last_known_addr { host: "127.24.0.188" port: 37741 } attrs { promote: false } } }
I20260812 01:19:28.790520 4243 mvcc.cc:205] Tried to move back new op lower bound from 7317494041756844032 to 7317494041553625088. Current Snapshot: MvccSnapshot[applied={T|T < 7317494041756844032}]
I20260812 01:19:28.790551 4228 sys_catalog.cc:458] T 00000000000000000000000000000000 P ccb9bfbd91334a59bb9b562bc5be7fa6 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 01:19:28.795405 4230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:28.792367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.130:45783 (local address 127.0.0.1:32902)
0812 01:19:28.792478 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:28.792488 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:28.792527 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:28.792723 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:28.792724 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:28.792726 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:19:28.792778 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:28.792781 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:28.793719 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:28.793720 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:28.794450 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:28.794452 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:28.795210 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:28.795211 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:28.795269 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:28.795275 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 01:19:28.795305 (+ 30us) client_negotiation.cc:240] Negotiation successful
0812 01:19:28.795342 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94}
I20260812 01:19:28.795753 4250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:28.792448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32902 (local address 127.24.0.130:45783)
0812 01:19:28.792554 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:28.792556 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:28.792564 (+ 8us) server_negotiation.cc:407] Connection header received
0812 01:19:28.792592 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:28.792593 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:28.792605 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:28.792701 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:19:28.792858 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:28.793638 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:28.794536 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:28.795123 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:28.795597 (+ 474us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:28.795613 (+ 16us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:28.795633 (+ 20us) server_negotiation.cc:299] Negotiation successful
0812 01:19:28.795673 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85,"thread_start_us":56,"threads_started":1}
I20260812 01:19:28.795972 3754 tablet_service.cc:1467] Processing CreateTablet for tablet ce4ba79ff8504863b95be8a775e01dc2 (DEFAULT_TABLE table=test-table [id=06fdc6d55e98445eabbd546f088925e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:19:28.796021 4257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:28.792451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.129:34201 (local address 127.0.0.1:33818)
0812 01:19:28.793046 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:28.793061 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:28.793120 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:28.793315 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:28.793315 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:28.793318 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:19:28.793371 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:28.793372 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:28.794064 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:28.794065 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:28.794992 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:28.794994 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:28.795798 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:28.795798 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:28.795861 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:28.795865 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 01:19:28.795919 (+ 54us) client_negotiation.cc:240] Negotiation successful
0812 01:19:28.795958 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":578,"mutex_wait_us":413,"thread_start_us":70,"threads_started":1}
I20260812 01:19:28.796053 4253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:28.792632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33818 (local address 127.24.0.129:34201)
0812 01:19:28.792824 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:28.792826 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:28.793112 (+ 286us) server_negotiation.cc:407] Connection header received
0812 01:19:28.793204 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:28.793206 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:28.793216 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:28.793253 (+ 37us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:19:28.793439 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:28.793992 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:28.795096 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:28.795684 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:28.795725 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:28.795736 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:28.795938 (+ 202us) server_negotiation.cc:299] Negotiation successful
0812 01:19:28.795979 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":67,"threads_started":1}
I20260812 01:19:28.796172 3754 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce4ba79ff8504863b95be8a775e01dc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:19:28.796325 3516 tablet_service.cc:1467] Processing CreateTablet for tablet ce4ba79ff8504863b95be8a775e01dc2 (DEFAULT_TABLE table=test-table [id=06fdc6d55e98445eabbd546f088925e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:19:28.796476 3516 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce4ba79ff8504863b95be8a775e01dc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:19:28.797628 4259 tablet_bootstrap.cc:492] T ce4ba79ff8504863b95be8a775e01dc2 P 06e0895a4bc34d5dad8a872a6fe475eb: Bootstrap starting.
I20260812 01:19:28.797755 4258 tablet_bootstrap.cc:492] T ce4ba79ff8504863b95be8a775e01dc2 P d645f2e569134a37baea2860781c5e0e: Bootstrap starting.
I20260812 01:19:28.797968 4229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:28.792448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.131:41837 (local address 127.0.0.1:59788)
0812 01:19:28.792545 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:28.792554 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:28.792605 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:28.793033 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:28.793034 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:28.793037 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:19:28.793083 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:28.793085 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:28.794525 (+ 1440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:28.794526 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:28.795365 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:28.795367 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:28.797748 (+ 2381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:28.797749 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:28.797813 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:28.797816 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 01:19:28.797868 (+ 52us) client_negotiation.cc:240] Negotiation successful
0812 01:19:28.797907 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":82}
I20260812 01:19:28.797986 4255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:28.792756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59788 (local address 127.24.0.131:41837)
0812 01:19:28.792897 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:28.792899 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:28.792907 (+ 8us) server_negotiation.cc:407] Connection header received
0812 01:19:28.792932 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:28.792932 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:28.792943 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:28.793029 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:19:28.793147 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:28.793729 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:28.795451 (+ 1722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:28.797651 (+ 2200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:28.797736 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:28.797750 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:28.797874 (+ 124us) server_negotiation.cc:299] Negotiation successful
0812 01:19:28.797911 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":91,"threads_started":1}
I20260812 01:19:28.798235 4258 tablet_bootstrap.cc:654] T ce4ba79ff8504863b95be8a775e01dc2 P d645f2e569134a37baea2860781c5e0e: Neither blocks nor log segments found. Creating new log.
I20260812 01:19:28.798269 3989 tablet_service.cc:1467] Processing CreateTablet for tablet ce4ba79ff8504863b95be8a775e01dc2 (DEFAULT_TABLE table=test-table [id=06fdc6d55e98445eabbd546f088925e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:19:28.798398 3989 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce4ba79ff8504863b95be8a775e01dc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:19:28.798481 4259 tablet_bootstrap.cc:654] T ce4ba79ff8504863b95be8a775e01dc2 P 06e0895a4bc34d5dad8a872a6fe475eb: Neither blocks nor log segments found. Creating new log.
I20260812 01:19:28.798842 4258 tablet_bootstrap.cc:492] T ce4ba79ff8504863b95be8a775e01dc2 P d645f2e569134a37baea2860781c5e0e: No bootstrap required, opened a new log
I20260812 01:19:28.798938 4258 ts_tablet_manager.cc:1397] T ce4ba79ff8504863b95be8a775e01dc2 P d645f2e569134a37baea2860781c5e0e: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 01:19:28.799062 4258 raft_consensus.cc:348] T ce4ba79ff8504863b95be8a775e01dc2 P d645f2e569134a37baea2860781c5e0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d645f2e569134a37baea2860781c5e0e" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 45783 } } peers { permanent_uuid: "06e0895a4bc34d5dad8a872a6fe475eb" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 34201 } } peers { permanent_uuid: "85a369874676488a8c43c1a2cf51a95a" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 41837 } }
I20260812 01:19:28.799137 4259 tablet_bootstrap.cc:492] T ce4ba79ff8504863b95be8a775e01dc2 P 06e0895a4bc34d5dad8a872a6fe475eb: No bootstrap required, opened a new log
I20260812 01:19:28.799149 4258 raft_consensus.cc:374] T ce4ba79ff8504863b95be8a775e01dc2 P d645f2e569134a37baea2860781c5e0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:19:28.799180 4258 raft_consensus.cc:729] T ce4ba79ff8504863b95be8a775e01dc2 P d645f2e569134a37baea2860781c5e0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d645f2e569134a37baea2860781c5e0e, State: Initialized, Role: FOLLOWER
I20260812 01:19:28.799196 4259 ts_tablet_manager.cc:1397] T ce4ba79ff8504863b95be8a775e01dc2 P 06e0895a4bc34d5dad8a872a6fe475eb: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 01:19:28.799242 4258 consensus_queue.cc:260] T ce4ba79ff8504863b95be8a775e01dc2 P d645f2e569134a37baea2860781c5e0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d645f2e569134a37baea2860781c5e0e" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 45783 } } peers { permanent_uuid: "06e0895a4bc34d5dad8a872a6fe475eb" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 34201 } } peers { permanent_uuid: "85a369874676488a8c43c1a2cf51a95a" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 41837 } }
I20260812 01:19:28.799471 4258 ts_tablet_manager.cc:1428] T ce4ba79ff8504863b95be8a775e01dc2 P d645f2e569134a37baea2860781c5e0e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:28.799353 4259 raft_consensus.cc:348] T ce4ba79ff8504863b95be8a775e01dc2 P 06e0895a4bc34d5dad8a872a6fe475eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d645f2e569134a37baea2860781c5e0e" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 45783 } } peers { permanent_uuid: "06e0895a4bc34d5dad8a872a6fe475eb" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 34201 } } peers { permanent_uuid: "85a369874676488a8c43c1a2cf51a95a" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 41837 } }
I20260812 01:19:28.799861 4259 raft_consensus.cc:374] T ce4ba79ff8504863b95be8a775e01dc2 P 06e0895a4bc34d5dad8a872a6fe475eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:19:28.799907 4259 raft_consensus.cc:729] T ce4ba79ff8504863b95be8a775e01dc2 P 06e0895a4bc34d5dad8a872a6fe475eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06e0895a4bc34d5dad8a872a6fe475eb, State: Initialized, Role: FOLLOWER
I20260812 01:19:28.799827 4266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:28.799603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:46409 (local address 127.24.0.130:58875)
0812 01:19:28.799751 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:28.799792 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:46409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":86,"threads_started":1}
I20260812 01:19:28.799986 4259 consensus_queue.cc:260] T ce4ba79ff8504863b95be8a775e01dc2 P 06e0895a4bc34d5dad8a872a6fe475eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d645f2e569134a37baea2860781c5e0e" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 45783 } } peers { permanent_uuid: "06e0895a4bc34d5dad8a872a6fe475eb" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 34201 } } peers { permanent_uuid: "85a369874676488a8c43c1a2cf51a95a" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 41837 } }
I20260812 01:19:28.800025 4267 tablet_bootstrap.cc:492] T ce4ba79ff8504863b95be8a775e01dc2 P 85a369874676488a8c43c1a2cf51a95a: Bootstrap starting.
I20260812 01:19:28.800231 3870 heartbeater.cc:503] Master 127.24.0.189:38795 was elected leader, sending a full tablet report...
I20260812 01:19:28.800416 4267 tablet_bootstrap.cc:654] T ce4ba79ff8504863b95be8a775e01dc2 P 85a369874676488a8c43c1a2cf51a95a: Neither blocks nor log segments found. Creating new log.
I20260812 01:19:28.800419 4259 ts_tablet_manager.cc:1428] T ce4ba79ff8504863b95be8a775e01dc2 P 06e0895a4bc34d5dad8a872a6fe475eb: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 01:19:28.800944 4266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:28.800292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:46409 (local address 127.24.0.130:36989)
0812 01:19:28.800706 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:28.800736 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:46409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":399}
I20260812 01:19:28.801254 3631 heartbeater.cc:503] Master 127.24.0.189:38795 was elected leader, sending a full tablet report...
I20260812 01:19:28.801343 4267 tablet_bootstrap.cc:492] T ce4ba79ff8504863b95be8a775e01dc2 P 85a369874676488a8c43c1a2cf51a95a: No bootstrap required, opened a new log
I20260812 01:19:28.801396 4267 ts_tablet_manager.cc:1397] T ce4ba79ff8504863b95be8a775e01dc2 P 85a369874676488a8c43c1a2cf51a95a: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260812 01:19:28.801512 4267 raft_consensus.cc:348] T ce4ba79ff8504863b95be8a775e01dc2 P 85a369874676488a8c43c1a2cf51a95a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d645f2e569134a37baea2860781c5e0e" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 45783 } } peers { permanent_uuid: "06e0895a4bc34d5dad8a872a6fe475eb" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 34201 } } peers { permanent_uuid: "85a369874676488a8c43c1a2cf51a95a" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 41837 } }
I20260812 01:19:28.801604 4267 raft_consensus.cc:374] T ce4ba79ff8504863b95be8a775e01dc2 P 85a369874676488a8c43c1a2cf51a95a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:19:28.801635 4267 raft_consensus.cc:729] T ce4ba79ff8504863b95be8a775e01dc2 P 85a369874676488a8c43c1a2cf51a95a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85a369874676488a8c43c1a2cf51a95a, State: Initialized, Role: FOLLOWER
I20260812 01:19:28.801703 4267 consensus_queue.cc:260] T ce4ba79ff8504863b95be8a775e01dc2 P 85a369874676488a8c43c1a2cf51a95a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d645f2e569134a37baea2860781c5e0e" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 45783 } } peers { permanent_uuid: "06e0895a4bc34d5dad8a872a6fe475eb" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 34201 } } peers { permanent_uuid: "85a369874676488a8c43c1a2cf51a95a" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 41837 } }
I20260812 01:19:28.801884 4267 ts_tablet_manager.cc:1428] T ce4ba79ff8504863b95be8a775e01dc2 P 85a369874676488a8c43c1a2cf51a95a: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260812 01:19:28.802157 4271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:28.800877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:46409 (local address 127.24.0.129:49093)
0812 01:19:28.802068 (+ 1191us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:28.802107 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:46409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1098,"thread_start_us":79,"threads_started":1}
I20260812 01:19:28.802366 4274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:28.802087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:46409 (local address 127.24.0.131:36261)
0812 01:19:28.802244 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:28.802284 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:46409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":85,"threads_started":1}
I20260812 01:19:28.802527 4271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:28.802414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:46409 (local address 127.24.0.129:34297)
0812 01:19:28.802444 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:28.802502 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:46409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
I20260812 01:19:28.802901 4274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:28.802799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:46409 (local address 127.24.0.131:41875)
0812 01:19:28.802837 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:28.802870 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:46409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260812 01:19:28.812038 4268 raft_consensus.cc:482] T ce4ba79ff8504863b95be8a775e01dc2 P 06e0895a4bc34d5dad8a872a6fe475eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 01:19:28.812171 4268 raft_consensus.cc:504] T ce4ba79ff8504863b95be8a775e01dc2 P 06e0895a4bc34d5dad8a872a6fe475eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d645f2e569134a37baea2860781c5e0e" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 45783 } } peers { permanent_uuid: "06e0895a4bc34d5dad8a872a6fe475eb" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 34201 } } peers { permanent_uuid: "85a369874676488a8c43c1a2cf51a95a" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 41837 } }
I20260812 01:19:28.812417 4268 leader_election.cc:288] T ce4ba79ff8504863b95be8a775e01dc2 P 06e0895a4bc34d5dad8a872a6fe475eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d645f2e569134a37baea2860781c5e0e (127.24.0.130:45783), 85a369874676488a8c43c1a2cf51a95a (127.24.0.131:41837)
I20260812 01:19:28.815351 4255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:28.812726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.129:44329 (local address 127.24.0.131:41837)
0812 01:19:28.812782 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:28.812784 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:28.812893 (+ 109us) server_negotiation.cc:407] Connection header received
0812 01:19:28.812949 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:28.812950 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:28.812961 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:28.813036 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:19:28.813170 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:28.813727 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:28.814671 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:28.815008 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:28.815072 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:28.815106 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:28.815248 (+ 142us) server_negotiation.cc:299] Negotiation successful
0812 01:19:28.815292 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 01:19:28.815347 4275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:28.812685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.131:41837 (local address 127.24.0.129:44329)
0812 01:19:28.812829 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:28.812838 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:28.812899 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:28.813045 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:28.813046 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:28.813048 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:19:28.813108 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:28.813109 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:28.813819 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:28.813820 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:28.814570 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:28.814572 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:28.815096 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:28.815097 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:28.815179 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:28.815182 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 01:19:28.815225 (+ 43us) client_negotiation.cc:240] Negotiation successful
0812 01:19:28.815253 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":90,"threads_started":1}
I20260812 01:19:28.815442 4250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:28.812646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.129:44055 (local address 127.24.0.130:45783)
0812 01:19:28.812685 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:28.812687 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:28.812695 (+ 8us) server_negotiation.cc:407] Connection header received
0812 01:19:28.812791 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:28.812791 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:28.812805 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:28.812863 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:19:28.813059 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:28.813650 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:28.814639 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:28.815022 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:28.815114 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:28.815123 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:28.815326 (+ 203us) server_negotiation.cc:299] Negotiation successful
0812 01:19:28.815371 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 01:19:28.815459 4271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:28.812573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.130:45783 (local address 127.24.0.129:44055)
0812 01:19:28.812607 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:28.812617 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:28.812668 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:28.812898 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:28.812900 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:28.812903 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:19:28.812971 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:28.812973 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:28.813739 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:28.813740 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:28.814548 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:28.814551 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:28.815157 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:28.815158 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:28.815245 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:28.815248 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 01:19:28.815304 (+ 56us) client_negotiation.cc:240] Negotiation successful
0812 01:19:28.815343 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16,"mutex_wait_us":20}
I20260812 01:19:28.815969 4022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce4ba79ff8504863b95be8a775e01dc2" candidate_uuid: "06e0895a4bc34d5dad8a872a6fe475eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85a369874676488a8c43c1a2cf51a95a" is_pre_election: true
I20260812 01:19:28.816052 4022 raft_consensus.cc:2454] T ce4ba79ff8504863b95be8a775e01dc2 P 85a369874676488a8c43c1a2cf51a95a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06e0895a4bc34d5dad8a872a6fe475eb in term 0.
I20260812 01:19:28.815994 3785 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce4ba79ff8504863b95be8a775e01dc2" candidate_uuid: "06e0895a4bc34d5dad8a872a6fe475eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d645f2e569134a37baea2860781c5e0e" is_pre_election: true
I20260812 01:19:28.816166 3785 raft_consensus.cc:2454] T ce4ba79ff8504863b95be8a775e01dc2 P d645f2e569134a37baea2860781c5e0e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06e0895a4bc34d5dad8a872a6fe475eb in term 0.
I20260812 01:19:28.816274 3437 leader_election.cc:302] T ce4ba79ff8504863b95be8a775e01dc2 P 06e0895a4bc34d5dad8a872a6fe475eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06e0895a4bc34d5dad8a872a6fe475eb, 85a369874676488a8c43c1a2cf51a95a; no voters:
I20260812 01:19:28.816418 4268 raft_consensus.cc:2781] T ce4ba79ff8504863b95be8a775e01dc2 P 06e0895a4bc34d5dad8a872a6fe475eb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 01:19:28.816455 4268 raft_consensus.cc:482] T ce4ba79ff8504863b95be8a775e01dc2 P 06e0895a4bc34d5dad8a872a6fe475eb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 01:19:28.816470 4268 raft_consensus.cc:3037] T ce4ba79ff8504863b95be8a775e01dc2 P 06e0895a4bc34d5dad8a872a6fe475eb [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:19:28.817097 4268 raft_consensus.cc:504] T ce4ba79ff8504863b95be8a775e01dc2 P 06e0895a4bc34d5dad8a872a6fe475eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d645f2e569134a37baea2860781c5e0e" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 45783 } } peers { permanent_uuid: "06e0895a4bc34d5dad8a872a6fe475eb" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 34201 } } peers { permanent_uuid: "85a369874676488a8c43c1a2cf51a95a" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 41837 } }
I20260812 01:19:28.817231 4268 leader_election.cc:288] T ce4ba79ff8504863b95be8a775e01dc2 P 06e0895a4bc34d5dad8a872a6fe475eb [CANDIDATE]: Term 1 election: Requested vote from peers d645f2e569134a37baea2860781c5e0e (127.24.0.130:45783), 85a369874676488a8c43c1a2cf51a95a (127.24.0.131:41837)
I20260812 01:19:28.817355 3785 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce4ba79ff8504863b95be8a775e01dc2" candidate_uuid: "06e0895a4bc34d5dad8a872a6fe475eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d645f2e569134a37baea2860781c5e0e"
I20260812 01:19:28.817385 4022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce4ba79ff8504863b95be8a775e01dc2" candidate_uuid: "06e0895a4bc34d5dad8a872a6fe475eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85a369874676488a8c43c1a2cf51a95a"
I20260812 01:19:28.817433 3785 raft_consensus.cc:3037] T ce4ba79ff8504863b95be8a775e01dc2 P d645f2e569134a37baea2860781c5e0e [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:19:28.817445 4022 raft_consensus.cc:3037] T ce4ba79ff8504863b95be8a775e01dc2 P 85a369874676488a8c43c1a2cf51a95a [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:19:28.818229 4022 raft_consensus.cc:2454] T ce4ba79ff8504863b95be8a775e01dc2 P 85a369874676488a8c43c1a2cf51a95a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06e0895a4bc34d5dad8a872a6fe475eb in term 1.
I20260812 01:19:28.818333 3785 raft_consensus.cc:2454] T ce4ba79ff8504863b95be8a775e01dc2 P d645f2e569134a37baea2860781c5e0e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06e0895a4bc34d5dad8a872a6fe475eb in term 1.
I20260812 01:19:28.818395 3437 leader_election.cc:302] T ce4ba79ff8504863b95be8a775e01dc2 P 06e0895a4bc34d5dad8a872a6fe475eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06e0895a4bc34d5dad8a872a6fe475eb, 85a369874676488a8c43c1a2cf51a95a; no voters:
I20260812 01:19:28.818487 4268 raft_consensus.cc:2781] T ce4ba79ff8504863b95be8a775e01dc2 P 06e0895a4bc34d5dad8a872a6fe475eb [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:19:28.818615 4268 raft_consensus.cc:686] T ce4ba79ff8504863b95be8a775e01dc2 P 06e0895a4bc34d5dad8a872a6fe475eb [term 1 LEADER]: Becoming Leader. State: Replica: 06e0895a4bc34d5dad8a872a6fe475eb, State: Running, Role: LEADER
I20260812 01:19:28.818699 4268 consensus_queue.cc:237] T ce4ba79ff8504863b95be8a775e01dc2 P 06e0895a4bc34d5dad8a872a6fe475eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d645f2e569134a37baea2860781c5e0e" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 45783 } } peers { permanent_uuid: "06e0895a4bc34d5dad8a872a6fe475eb" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 34201 } } peers { permanent_uuid: "85a369874676488a8c43c1a2cf51a95a" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 41837 } }
I20260812 01:19:28.818904 4271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:28.818803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:46409 (local address 127.24.0.129:35241)
0812 01:19:28.818855 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:28.818879 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:46409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260812 01:19:28.819443 3031 catalog_manager.cc:5495] T ce4ba79ff8504863b95be8a775e01dc2 P 06e0895a4bc34d5dad8a872a6fe475eb reported cstate change: term changed from 0 to 1, leader changed from <none> to 06e0895a4bc34d5dad8a872a6fe475eb (127.24.0.129). New cstate: current_term: 1 leader_uuid: "06e0895a4bc34d5dad8a872a6fe475eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d645f2e569134a37baea2860781c5e0e" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 45783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06e0895a4bc34d5dad8a872a6fe475eb" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 34201 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85a369874676488a8c43c1a2cf51a95a" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 41837 } health_report { overall_health: UNKNOWN } } }
I20260812 01:19:28.831974 4253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:28.829603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33828 (local address 127.24.0.129:34201)
0812 01:19:28.829647 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:28.829649 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:28.829657 (+ 8us) server_negotiation.cc:407] Connection header received
0812 01:19:28.829747 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:28.829748 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:28.829775 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:28.829854 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:19:28.829968 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:28.830430 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:28.831172 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:28.831364 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:28.831441 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:28.831636 (+ 195us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:19:28.831733 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:19:28.831812 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:28.831869 (+ 57us) server_negotiation.cc:299] Negotiation successful
0812 01:19:28.831890 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 01:19:28.833086 4022 raft_consensus.cc:1261] T ce4ba79ff8504863b95be8a775e01dc2 P 85a369874676488a8c43c1a2cf51a95a [term 1 FOLLOWER]: Refusing update from remote peer 06e0895a4bc34d5dad8a872a6fe475eb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:19:28.833192 3785 raft_consensus.cc:1261] T ce4ba79ff8504863b95be8a775e01dc2 P d645f2e569134a37baea2860781c5e0e [term 1 FOLLOWER]: Refusing update from remote peer 06e0895a4bc34d5dad8a872a6fe475eb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:19:28.833359 4268 consensus_queue.cc:1035] T ce4ba79ff8504863b95be8a775e01dc2 P 06e0895a4bc34d5dad8a872a6fe475eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "85a369874676488a8c43c1a2cf51a95a" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 41837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:19:28.833528 4276 consensus_queue.cc:1035] T ce4ba79ff8504863b95be8a775e01dc2 P 06e0895a4bc34d5dad8a872a6fe475eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d645f2e569134a37baea2860781c5e0e" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 45783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:19:28.834039 4102 heartbeater.cc:503] Master 127.24.0.189:38795 was elected leader, sending a full tablet report...
I20260812 01:19:28.834489 4266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:28.834353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:46409 (local address 127.24.0.130:40039)
0812 01:19:28.834433 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:28.834460 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:46409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260812 01:19:28.834489 4274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:28.834364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:46409 (local address 127.24.0.131:49271)
0812 01:19:28.834433 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:28.834460 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:46409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260812 01:19:28.835867 4274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:28.835753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:46409 (local address 127.24.0.131:39397)
0812 01:19:28.835815 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:28.835840 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:46409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260812 01:19:28.836090 4266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:28.835955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:46409 (local address 127.24.0.130:49629)
0812 01:19:28.836017 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:28.836041 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:46409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260812 01:19:28.836529 4271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:28.836429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:46409 (local address 127.24.0.129:47297)
0812 01:19:28.836462 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:28.836498 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:46409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
I20260812 01:19:28.836627 4285 mvcc.cc:205] Tried to move back new op lower bound from 7317494041938149376 to 7317494041881944064. Current Snapshot: MvccSnapshot[applied={T|T < 7317494041938149376}]
I20260812 01:19:28.838786 3030 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44146:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:19:28.838883 3030 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44146:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:19:28.840322 3030 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 35a50d99aa1b4d9c8a4b11849067ad0f: Sending DeleteTablet for 3 replicas of tablet ce4ba79ff8504863b95be8a775e01dc2
I20260812 01:19:28.840612 3989 tablet_service.cc:1514] Processing DeleteTablet for tablet ce4ba79ff8504863b95be8a775e01dc2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:19:28 UTC) from {username='oryx', principal='oryx/127.24.0.189@KRBTEST.COM'} at 127.0.0.1:59788
I20260812 01:19:28.840698 3516 tablet_service.cc:1514] Processing DeleteTablet for tablet ce4ba79ff8504863b95be8a775e01dc2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:19:28 UTC) from {username='oryx', principal='oryx/127.24.0.189@KRBTEST.COM'} at 127.0.0.1:33818
I20260812 01:19:28.840807 3754 tablet_service.cc:1514] Processing DeleteTablet for tablet ce4ba79ff8504863b95be8a775e01dc2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:19:28 UTC) from {username='oryx', principal='oryx/127.24.0.189@KRBTEST.COM'} at 127.0.0.1:32902
I20260812 01:19:28.840943 4297 tablet_replica.cc:330] T ce4ba79ff8504863b95be8a775e01dc2 P 06e0895a4bc34d5dad8a872a6fe475eb: stopping tablet replica
I20260812 01:19:28.841005 4297 raft_consensus.cc:2229] T ce4ba79ff8504863b95be8a775e01dc2 P 06e0895a4bc34d5dad8a872a6fe475eb [term 1 LEADER]: Raft consensus shutting down.
I20260812 01:19:28.841084 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 3410
I20260812 01:19:28.849514 4298 tablet_replica.cc:330] T ce4ba79ff8504863b95be8a775e01dc2 P d645f2e569134a37baea2860781c5e0e: stopping tablet replica
I20260812 01:19:28.849615 4298 raft_consensus.cc:2229] T ce4ba79ff8504863b95be8a775e01dc2 P d645f2e569134a37baea2860781c5e0e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 01:19:28.849679 4298 raft_consensus.cc:2258] T ce4ba79ff8504863b95be8a775e01dc2 P d645f2e569134a37baea2860781c5e0e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:19:28.850703 4295 tablet_replica.cc:330] T ce4ba79ff8504863b95be8a775e01dc2 P 85a369874676488a8c43c1a2cf51a95a: stopping tablet replica
I20260812 01:19:28.850790 4295 raft_consensus.cc:2229] T ce4ba79ff8504863b95be8a775e01dc2 P 85a369874676488a8c43c1a2cf51a95a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 01:19:28.850869 4295 raft_consensus.cc:2258] T ce4ba79ff8504863b95be8a775e01dc2 P 85a369874676488a8c43c1a2cf51a95a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:19:28.851146 4298 ts_tablet_manager.cc:1905] T ce4ba79ff8504863b95be8a775e01dc2 P d645f2e569134a37baea2860781c5e0e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:19:28.852329 4295 ts_tablet_manager.cc:1905] T ce4ba79ff8504863b95be8a775e01dc2 P 85a369874676488a8c43c1a2cf51a95a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:19:28.853302 4298 ts_tablet_manager.cc:1918] T ce4ba79ff8504863b95be8a775e01dc2 P d645f2e569134a37baea2860781c5e0e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 01:19:28.853374 4298 log.cc:1196] T ce4ba79ff8504863b95be8a775e01dc2 P d645f2e569134a37baea2860781c5e0e: Deleting WAL directory at /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/ts-1/wal/wals/ce4ba79ff8504863b95be8a775e01dc2
I20260812 01:19:28.853643 4298 ts_tablet_manager.cc:1939] T ce4ba79ff8504863b95be8a775e01dc2 P d645f2e569134a37baea2860781c5e0e: Deleting consensus metadata
I20260812 01:19:28.853653 4295 ts_tablet_manager.cc:1918] T ce4ba79ff8504863b95be8a775e01dc2 P 85a369874676488a8c43c1a2cf51a95a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 01:19:28.853730 4295 log.cc:1196] T ce4ba79ff8504863b95be8a775e01dc2 P 85a369874676488a8c43c1a2cf51a95a: Deleting WAL directory at /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786497534195564-24578-0/minicluster-data/ts-2/wal/wals/ce4ba79ff8504863b95be8a775e01dc2
I20260812 01:19:28.853994 4295 ts_tablet_manager.cc:1939] T ce4ba79ff8504863b95be8a775e01dc2 P 85a369874676488a8c43c1a2cf51a95a: Deleting consensus metadata
I20260812 01:19:28.854029 2999 catalog_manager.cc:4841] TS d645f2e569134a37baea2860781c5e0e (127.24.0.130:45783): tablet ce4ba79ff8504863b95be8a775e01dc2 (table test-table [id=06fdc6d55e98445eabbd546f088925e4]) successfully deleted
I20260812 01:19:28.854380 2996 catalog_manager.cc:4841] TS 85a369874676488a8c43c1a2cf51a95a (127.24.0.131:41837): tablet ce4ba79ff8504863b95be8a775e01dc2 (table test-table [id=06fdc6d55e98445eabbd546f088925e4]) successfully deleted
I20260812 01:19:28.856920 4229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:28.856805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.129:34201 (local address 127.0.0.1:33832)
0812 01:19:28.856864 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:28.856894 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.129:34201: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260812 01:19:28.856997 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 3641
W20260812 01:19:28.857115 2995 catalog_manager.cc:4584] TS 06e0895a4bc34d5dad8a872a6fe475eb (127.24.0.129:34201): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ce4ba79ff8504863b95be8a775e01dc2: Network error: Client connection negotiation failed: client connection to 127.24.0.129:34201: connect: Connection refused (error 111)
I20260812 01:19:28.872629 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 3877
I20260812 01:19:28.885339 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 2791
I20260812 01:19:28.900424 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 3123
2026-08-12T01:19:28Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (5174 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 01:19:28 dist-test-slave-t3q3 krb5kdc[4301](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 01:19:28 dist-test-slave-t3q3 krb5kdc[4301](info): set up 2 sockets
Aug 12 01:19:28 dist-test-slave-t3q3 krb5kdc[4301](info): commencing operation
krb5kdc: starting...
W20260812 01:19:30.163719 24578 mini_kdc.cc:121] Time spent starting KDC: real 1.231s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/krb5kdc/test-user.keytab.
Aug 12 01:19:30 dist-test-slave-t3q3 krb5kdc[4301](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497570, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T01:19:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:19:30Z Disabled control of system clock
WARNING: no policy specified for oryx/127.24.0.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.0.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.0.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/krb5kdc/oryx_127.24.0.190.keytab.
Entry for principal oryx/127.24.0.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/krb5kdc/oryx_127.24.0.190.keytab.
I20260812 01:19:30.291352 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.190:39395
--webserver_interface=127.24.0.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/krb5kdc/oryx_127.24.0.190.keytab
--principal=oryx/127.24.0.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:45119
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.0.190:39395
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:19:30.373697 4322 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:19:30.373898 4322 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:19:30.373919 4322 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:19:30.374964 4322 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:19:30.375000 4322 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260812 01:19:30.375012 4322 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:19:30.375025 4322 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:19:30.375037 4322 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:19:30.375049 4322 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:19:30.376505 4322 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:45119
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.0.190:39395
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/krb5kdc/oryx_127.24.0.190.keytab
--principal=oryx/127.24.0.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.190:39395
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.0.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4322
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:19:30.376694 4322 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:19:30.376860 4322 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:19:30.381525 4336 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:30.381525 4334 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:30.381506 4329 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:19:30.381799 4322 server_base.cc:1034] running on GCE node
I20260812 01:19:30.382053 4322 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:19:30.382409 4322 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:19:30.383577 4322 hybrid_clock.cc:648] HybridClock initialized: now 1786497570383571 us; error 39 us; skew 500 ppm
Aug 12 01:19:30 dist-test-slave-t3q3 krb5kdc[4301](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497570, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.0.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:19:30.391870 4322 init.cc:377] Logged in from keytab as oryx/127.24.0.190@KRBTEST.COM (short username oryx)
I20260812 01:19:30.392567 4322 webserver.cc:460] Webserver started at http://127.24.0.190:43643/ using document root <none> and password file <none>
I20260812 01:19:30.392933 4322 fs_manager.cc:359] Metadata directory not provided
I20260812 01:19:30.393028 4322 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:19:30.393190 4322 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:19:30.394263 4322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/minicluster-data/master-0/data/instance:
uuid: "8cf578e52d544713ab6c35cb1333039f"
format_stamp: "Formatted at 2026-08-12 01:19:30 on dist-test-slave-t3q3"
server_key: "c475ba7988a56237b4f1580400551146"
server_key_iv: "934a2feecca98c9a623809135438bc70"
server_key_version: "encryptionkey@0"
I20260812 01:19:30.394649 4322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/minicluster-data/master-0/wal/instance:
uuid: "8cf578e52d544713ab6c35cb1333039f"
format_stamp: "Formatted at 2026-08-12 01:19:30 on dist-test-slave-t3q3"
server_key: "c475ba7988a56237b4f1580400551146"
server_key_iv: "934a2feecca98c9a623809135438bc70"
server_key_version: "encryptionkey@0"
I20260812 01:19:30.395991 4322 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 01:19:30.396775 4350 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:30.396939 4322 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 01:19:30.397032 4322 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/minicluster-data/master-0/wal
uuid: "8cf578e52d544713ab6c35cb1333039f"
format_stamp: "Formatted at 2026-08-12 01:19:30 on dist-test-slave-t3q3"
server_key: "c475ba7988a56237b4f1580400551146"
server_key_iv: "934a2feecca98c9a623809135438bc70"
server_key_version: "encryptionkey@0"
I20260812 01:19:30.397118 4322 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:19:30.409017 4322 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:19:30.410929 4322 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:19:30.411093 4322 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:19:30.415889 4322 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.190:39395
I20260812 01:19:30.415977 4450 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.190:39395 every 8 connection(s)
I20260812 01:19:30.416361 4322 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
I20260812 01:19:30.416507 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 4322
I20260812 01:19:30.416611 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/minicluster-data/master-0/wal/instance
I20260812 01:19:30.416872 24578 external_mini_cluster.cc:1428] Setting key ee5f9053a28f481d9edb722e2a7f3b6c
I20260812 01:19:30.417255 4454 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:19:30.420668 4454 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8cf578e52d544713ab6c35cb1333039f: Bootstrap starting.
I20260812 01:19:30.421317 4454 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8cf578e52d544713ab6c35cb1333039f: Neither blocks nor log segments found. Creating new log.
I20260812 01:19:30.421620 4454 log.cc:824] T 00000000000000000000000000000000 P 8cf578e52d544713ab6c35cb1333039f: Log is configured to *not* fsync() on all Append() calls
I20260812 01:19:30.422325 4454 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8cf578e52d544713ab6c35cb1333039f: No bootstrap required, opened a new log
Aug 12 01:19:30 dist-test-slave-t3q3 krb5kdc[4301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497570, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM
I20260812 01:19:30.424540 4454 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8cf578e52d544713ab6c35cb1333039f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cf578e52d544713ab6c35cb1333039f" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 39395 } }
I20260812 01:19:30.424700 4454 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8cf578e52d544713ab6c35cb1333039f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:19:30.424783 4454 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8cf578e52d544713ab6c35cb1333039f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8cf578e52d544713ab6c35cb1333039f, State: Initialized, Role: FOLLOWER
I20260812 01:19:30.424954 4454 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8cf578e52d544713ab6c35cb1333039f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cf578e52d544713ab6c35cb1333039f" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 39395 } }
I20260812 01:19:30.425052 4454 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8cf578e52d544713ab6c35cb1333039f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:19:30.425099 4454 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8cf578e52d544713ab6c35cb1333039f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:19:30.425148 4454 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8cf578e52d544713ab6c35cb1333039f [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:19:30.425930 4454 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8cf578e52d544713ab6c35cb1333039f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cf578e52d544713ab6c35cb1333039f" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 39395 } }
I20260812 01:19:30.426047 4454 leader_election.cc:302] T 00000000000000000000000000000000 P 8cf578e52d544713ab6c35cb1333039f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8cf578e52d544713ab6c35cb1333039f; no voters:
I20260812 01:19:30.426239 4454 leader_election.cc:288] T 00000000000000000000000000000000 P 8cf578e52d544713ab6c35cb1333039f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 01:19:30.426316 4464 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8cf578e52d544713ab6c35cb1333039f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:19:30.426476 4464 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8cf578e52d544713ab6c35cb1333039f [term 1 LEADER]: Becoming Leader. State: Replica: 8cf578e52d544713ab6c35cb1333039f, State: Running, Role: LEADER
I20260812 01:19:30.426491 4454 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8cf578e52d544713ab6c35cb1333039f [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:19:30.426617 4464 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8cf578e52d544713ab6c35cb1333039f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cf578e52d544713ab6c35cb1333039f" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 39395 } }
I20260812 01:19:30.426950 4464 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8cf578e52d544713ab6c35cb1333039f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8cf578e52d544713ab6c35cb1333039f. Latest consensus state: current_term: 1 leader_uuid: "8cf578e52d544713ab6c35cb1333039f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cf578e52d544713ab6c35cb1333039f" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 39395 } } }
I20260812 01:19:30.427018 4464 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8cf578e52d544713ab6c35cb1333039f [sys.catalog]: This master's current role is: LEADER
I20260812 01:19:30.427191 4466 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8cf578e52d544713ab6c35cb1333039f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8cf578e52d544713ab6c35cb1333039f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cf578e52d544713ab6c35cb1333039f" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 39395 } } }
I20260812 01:19:30.427244 4474 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:19:30.427261 4466 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8cf578e52d544713ab6c35cb1333039f [sys.catalog]: This master's current role is: LEADER
I20260812 01:19:30.427928 4474 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:19:30.428915 4462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:30.417430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44760 (local address 127.24.0.190:39395)
0812 01:19:30.417648 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:30.417657 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:30.417699 (+ 42us) server_negotiation.cc:407] Connection header received
0812 01:19:30.418389 (+ 690us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:30.418399 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:30.418464 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:30.418622 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:30.419241 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:30.420641 (+ 1400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:30.421450 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:30.421904 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:30.422023 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:30.424686 (+ 2663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:30.424689 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:30.424690 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:30.424700 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:30.426839 (+ 2139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:30.427336 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:30.427341 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:30.427341 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:30.427451 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:30.427738 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:30.427739 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:30.427740 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:30.428112 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:30.428235 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:30.428566 (+ 331us) server_negotiation.cc:299] Negotiation successful
0812 01:19:30.428705 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":86,"threads_started":1}
I20260812 01:19:30.433175 4474 catalog_manager.cc:1347] Generated new cluster ID: 05136bb4c02f40fba819909e6d486564
I20260812 01:19:30.433221 4474 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 01:19:30.452023 4474 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:19:30.452598 4474 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:19:30.469238 4474 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8cf578e52d544713ab6c35cb1333039f: Generated new TSK 0
I20260812 01:19:30.469439 4474 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.24.0.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.0.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.0.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/krb5kdc/oryx_127.24.0.129.keytab.
Entry for principal oryx/127.24.0.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/krb5kdc/oryx_127.24.0.129.keytab.
I20260812 01:19:30.507707 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.0.129:0
--local_ip_for_outbound_sockets=127.24.0.129
--webserver_interface=127.24.0.129
--webserver_port=0
--tserver_master_addrs=127.24.0.190:39395
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/krb5kdc/oryx_127.24.0.129.keytab
--principal=oryx/127.24.0.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:45119
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:19:30.590819 4497 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:19:30.591058 4497 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:19:30.591101 4497 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:19:30.592322 4497 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:19:30.592382 4497 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:19:30.592419 4497 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 01:19:30.592475 4497 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.0.129
I20260812 01:19:30.593989 4497 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:45119
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/krb5kdc/oryx_127.24.0.129.keytab
--principal=oryx/127.24.0.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.0.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.0.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.0.190:39395
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4497
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.0.129
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:19:30.594250 4497 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:19:30.594480 4497 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:19:30.599642 4503 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:30.599754 4506 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:30.599727 4504 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:19:30.599908 4497 server_base.cc:1034] running on GCE node
I20260812 01:19:30.600195 4497 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:19:30.600469 4497 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:19:30.601632 4497 hybrid_clock.cc:648] HybridClock initialized: now 1786497570601618 us; error 31 us; skew 500 ppm
Aug 12 01:19:30 dist-test-slave-t3q3 krb5kdc[4301](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497570, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.0.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:19:30.604205 4497 init.cc:377] Logged in from keytab as oryx/127.24.0.129@KRBTEST.COM (short username oryx)
I20260812 01:19:30.604626 4497 webserver.cc:460] Webserver started at http://127.24.0.129:42535/ using document root <none> and password file <none>
I20260812 01:19:30.604863 4497 fs_manager.cc:359] Metadata directory not provided
I20260812 01:19:30.604933 4497 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:19:30.605077 4497 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:19:30.606120 4497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/minicluster-data/ts-0/data/instance:
uuid: "0ffa5375dd2b431b852b9dd97c25d204"
format_stamp: "Formatted at 2026-08-12 01:19:30 on dist-test-slave-t3q3"
server_key: "0443cfae4d2ed3a711f19df8fcd3278a"
server_key_iv: "81cbef8c6a6dc5bd3302158e61b71484"
server_key_version: "encryptionkey@0"
I20260812 01:19:30.606496 4497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/minicluster-data/ts-0/wal/instance:
uuid: "0ffa5375dd2b431b852b9dd97c25d204"
format_stamp: "Formatted at 2026-08-12 01:19:30 on dist-test-slave-t3q3"
server_key: "0443cfae4d2ed3a711f19df8fcd3278a"
server_key_iv: "81cbef8c6a6dc5bd3302158e61b71484"
server_key_version: "encryptionkey@0"
I20260812 01:19:30.607839 4497 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 01:19:30.608655 4517 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:30.608837 4497 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:30.608939 4497 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/minicluster-data/ts-0/wal
uuid: "0ffa5375dd2b431b852b9dd97c25d204"
format_stamp: "Formatted at 2026-08-12 01:19:30 on dist-test-slave-t3q3"
server_key: "0443cfae4d2ed3a711f19df8fcd3278a"
server_key_iv: "81cbef8c6a6dc5bd3302158e61b71484"
server_key_version: "encryptionkey@0"
I20260812 01:19:30.609027 4497 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:19:30.631292 4497 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:19:30.633397 4497 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:19:30.633565 4497 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:19:30.634162 4497 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:19:30.634224 4497 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:30.634266 4497 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:19:30.634287 4497 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 01:19:30 dist-test-slave-t3q3 krb5kdc[4301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497570, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.0.129@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM
I20260812 01:19:30.641901 4497 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.129:33711
I20260812 01:19:30.642112 4701 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.129:33711 every 8 connection(s)
I20260812 01:19:30.642349 4497 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/minicluster-data/ts-0/data/info.pb
I20260812 01:19:30.643509 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 4497
I20260812 01:19:30.643636 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/minicluster-data/ts-0/wal/instance
I20260812 01:19:30.643891 24578 external_mini_cluster.cc:1428] Setting key 2e69e5846704f98d3bdbb7d2d6f90da0
I20260812 01:19:30.645022 4462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:30.634625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.129:37891 (local address 127.24.0.190:39395)
0812 01:19:30.634703 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:30.634706 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:30.634970 (+ 264us) server_negotiation.cc:407] Connection header received
0812 01:19:30.635746 (+ 776us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:30.635748 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:30.635759 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:30.635796 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:30.636808 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:30.637376 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:30.638426 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:30.638643 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:30.638723 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:30.641494 (+ 2771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:30.641496 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:30.641498 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:30.641511 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:30.643204 (+ 1693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:30.643751 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:30.643752 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:30.643753 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:30.643824 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:30.644169 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:30.644174 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:30.644174 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:30.644339 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:30.644408 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:30.644774 (+ 366us) server_negotiation.cc:299] Negotiation successful
0812 01:19:30.644882 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 01:19:30.645036 4527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:30.634608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:39395 (local address 127.24.0.129:37891)
0812 01:19:30.634817 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:30.634873 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:30.635618 (+ 745us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:30.635890 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:30.635894 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:30.636357 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:30.636677 (+ 320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:30.636686 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:30.637481 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:30.637482 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:30.638318 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:30.638321 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:30.638732 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:30.638733 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:30.638868 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:30.639412 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:30.639420 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:30.641383 (+ 1963us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:19:30.643277 (+ 1894us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:30.643283 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:30.643284 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:30.643650 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:30.643894 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:30.643895 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:30.643895 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:30.644051 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:30.644423 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:19:30.644430 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:19:30.644591 (+ 161us) client_negotiation.cc:769] Sending connection context
0812 01:19:30.644715 (+ 124us) client_negotiation.cc:240] Negotiation successful
0812 01:19:30.644872 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":96,"threads_started":1}
I20260812 01:19:30.645807 4703 heartbeater.cc:347] Connected to a master server at 127.24.0.190:39395
I20260812 01:19:30.645906 4703 heartbeater.cc:464] Registering TS with master...
I20260812 01:19:30.646103 4703 heartbeater.cc:511] Master 127.24.0.190:39395 requested a full tablet report, sending...
I20260812 01:19:30.646628 4372 ts_manager.cc:194] Registered new tserver with Master: 0ffa5375dd2b431b852b9dd97c25d204 (127.24.0.129:33711)
I20260812 01:19:30.647445 4372 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.0.129@KRBTEST.COM'} at 127.24.0.129:37891
I20260812 01:19:30.653185 4462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:30.646738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.129:42493 (local address 127.24.0.190:39395)
0812 01:19:30.646780 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:30.646782 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:30.647044 (+ 262us) server_negotiation.cc:407] Connection header received
0812 01:19:30.647105 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:30.647106 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:30.647118 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:30.647190 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:30.647582 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:30.648329 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:30.649358 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:30.649537 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:30.649587 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:30.650627 (+ 1040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:30.650628 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:30.650628 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:30.650634 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:30.652011 (+ 1377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:30.652445 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:30.652446 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:30.652446 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:30.652497 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:30.652732 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:30.652732 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:30.652732 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:30.652825 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:30.652874 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:30.653060 (+ 186us) server_negotiation.cc:299] Negotiation successful
0812 01:19:30.653118 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 01:19:30.653263 4715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:30.646810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:39395 (local address 127.24.0.129:42493)
0812 01:19:30.646954 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:30.646967 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:30.647036 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:30.647197 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:30.647198 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:30.647416 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:30.647488 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:30.647490 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:30.648450 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:30.648450 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:30.649266 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:30.649269 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:30.649609 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:30.649610 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:30.649684 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:30.650092 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:30.650097 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:30.650531 (+ 434us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:19:30.652118 (+ 1587us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:30.652119 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:30.652120 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:30.652366 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:30.652576 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:30.652578 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:30.652578 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:30.652655 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:30.652891 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:19:30.652891 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:19:30.652944 (+ 53us) client_negotiation.cc:769] Sending connection context
0812 01:19:30.653036 (+ 92us) client_negotiation.cc:240] Negotiation successful
0812 01:19:30.653153 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":90,"threads_started":1}
WARNING: no policy specified for oryx/127.24.0.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.0.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.0.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/krb5kdc/oryx_127.24.0.130.keytab.
Entry for principal oryx/127.24.0.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/krb5kdc/oryx_127.24.0.130.keytab.
I20260812 01:19:30.674772 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.0.130:0
--local_ip_for_outbound_sockets=127.24.0.130
--webserver_interface=127.24.0.130
--webserver_port=0
--tserver_master_addrs=127.24.0.190:39395
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/krb5kdc/oryx_127.24.0.130.keytab
--principal=oryx/127.24.0.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:45119
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:19:30.758636 4718 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:19:30.758822 4718 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:19:30.758845 4718 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:19:30.759968 4718 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:19:30.760007 4718 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:19:30.760027 4718 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 01:19:30.760044 4718 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.0.130
I20260812 01:19:30.761559 4718 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:45119
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/krb5kdc/oryx_127.24.0.130.keytab
--principal=oryx/127.24.0.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.0.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.0.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.0.190:39395
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4718
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.0.130
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:19:30.761737 4718 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:19:30.761905 4718 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:19:30.766723 4729 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:30.766816 4725 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:30.766723 4727 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:19:30.767005 4718 server_base.cc:1034] running on GCE node
I20260812 01:19:30.767207 4718 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:19:30.767388 4718 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:19:30.768549 4718 hybrid_clock.cc:648] HybridClock initialized: now 1786497570768535 us; error 42 us; skew 500 ppm
Aug 12 01:19:30 dist-test-slave-t3q3 krb5kdc[4301](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497570, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.0.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:19:30.771077 4718 init.cc:377] Logged in from keytab as oryx/127.24.0.130@KRBTEST.COM (short username oryx)
I20260812 01:19:30.771504 4718 webserver.cc:460] Webserver started at http://127.24.0.130:40925/ using document root <none> and password file <none>
I20260812 01:19:30.771744 4718 fs_manager.cc:359] Metadata directory not provided
I20260812 01:19:30.771816 4718 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:19:30.771970 4718 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:19:30.773017 4718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/minicluster-data/ts-1/data/instance:
uuid: "9b55292de1ce492b9e347c66f4cf0111"
format_stamp: "Formatted at 2026-08-12 01:19:30 on dist-test-slave-t3q3"
server_key: "456d76f952661ba20158caadb4d4668d"
server_key_iv: "5988ffc4e0a24888e5d737122aad10ad"
server_key_version: "encryptionkey@0"
I20260812 01:19:30.773386 4718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/minicluster-data/ts-1/wal/instance:
uuid: "9b55292de1ce492b9e347c66f4cf0111"
format_stamp: "Formatted at 2026-08-12 01:19:30 on dist-test-slave-t3q3"
server_key: "456d76f952661ba20158caadb4d4668d"
server_key_iv: "5988ffc4e0a24888e5d737122aad10ad"
server_key_version: "encryptionkey@0"
I20260812 01:19:30.774708 4718 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 01:19:30.775406 4742 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:30.775583 4718 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 01:19:30.775686 4718 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/minicluster-data/ts-1/wal
uuid: "9b55292de1ce492b9e347c66f4cf0111"
format_stamp: "Formatted at 2026-08-12 01:19:30 on dist-test-slave-t3q3"
server_key: "456d76f952661ba20158caadb4d4668d"
server_key_iv: "5988ffc4e0a24888e5d737122aad10ad"
server_key_version: "encryptionkey@0"
I20260812 01:19:30.775769 4718 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:19:30.795535 4718 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:19:30.800515 4718 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:19:30.802287 4718 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:19:30.804175 4718 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:19:30.804250 4718 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:30.804291 4718 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:19:30.804313 4718 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 01:19:30 dist-test-slave-t3q3 krb5kdc[4301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497570, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.0.130@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM
I20260812 01:19:30.811896 4718 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.130:39435
I20260812 01:19:30.811990 4920 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.130:39435 every 8 connection(s)
I20260812 01:19:30.812386 4718 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/minicluster-data/ts-1/data/info.pb
I20260812 01:19:30.814509 4462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:30.804903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.130:40585 (local address 127.24.0.190:39395)
0812 01:19:30.804959 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:30.804961 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:30.804973 (+ 12us) server_negotiation.cc:407] Connection header received
0812 01:19:30.805411 (+ 438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:30.805413 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:30.805425 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:30.805505 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:30.806462 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:30.807189 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:30.808293 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:30.808647 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:30.808737 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:30.811658 (+ 2921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:30.811660 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:30.811660 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:30.811668 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:30.813393 (+ 1725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:30.813761 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:30.813762 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:30.813762 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:30.813816 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:30.814012 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:30.814012 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:30.814012 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:30.814111 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:30.814160 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:30.814380 (+ 220us) server_negotiation.cc:299] Negotiation successful
0812 01:19:30.814446 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 01:19:30.814656 4755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:30.804534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:39395 (local address 127.24.0.130:40585)
0812 01:19:30.804690 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:30.804729 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:30.805303 (+ 574us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:30.805624 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:30.805630 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:30.806041 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:30.806335 (+ 294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:30.806342 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:30.807300 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:30.807301 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:30.808193 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:30.808195 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:30.808777 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:30.808778 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:30.808923 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:30.809576 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:30.809588 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:30.811541 (+ 1953us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:19:30.813464 (+ 1923us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:30.813470 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:30.813472 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:30.813693 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:30.813873 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:30.813874 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:30.813874 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:30.813960 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:30.814151 (+ 191us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:19:30.814154 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:19:30.814258 (+ 104us) client_negotiation.cc:769] Sending connection context
0812 01:19:30.814333 (+ 75us) client_negotiation.cc:240] Negotiation successful
0812 01:19:30.814447 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":77,"threads_started":1}
I20260812 01:19:30.815161 4921 heartbeater.cc:347] Connected to a master server at 127.24.0.190:39395
I20260812 01:19:30.815245 4921 heartbeater.cc:464] Registering TS with master...
I20260812 01:19:30.815423 4921 heartbeater.cc:511] Master 127.24.0.190:39395 requested a full tablet report, sending...
I20260812 01:19:30.815807 4375 ts_manager.cc:194] Registered new tserver with Master: 9b55292de1ce492b9e347c66f4cf0111 (127.24.0.130:39435)
I20260812 01:19:30.816322 4375 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.0.130@KRBTEST.COM'} at 127.24.0.130:40585
I20260812 01:19:30.821408 4462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:30.816295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.130:53291 (local address 127.24.0.190:39395)
0812 01:19:30.816338 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:30.816340 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:30.816453 (+ 113us) server_negotiation.cc:407] Connection header received
0812 01:19:30.816552 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:30.816554 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:30.816565 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:30.816623 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:30.817012 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:30.817448 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:30.818105 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:30.818274 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:30.818322 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:30.819055 (+ 733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:30.819056 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:30.819056 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:30.819062 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:30.820519 (+ 1457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:30.820810 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:30.820811 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:30.820811 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:30.820861 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:30.821014 (+ 153us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:30.821014 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:30.821014 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:30.821138 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:30.821186 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:30.821294 (+ 108us) server_negotiation.cc:299] Negotiation successful
0812 01:19:30.821352 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 01:19:30.821419 4928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:30.816207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:39395 (local address 127.24.0.130:53291)
0812 01:19:30.816387 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:30.816399 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:30.816480 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:30.816635 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:30.816636 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:30.816870 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:30.816936 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:30.816938 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:30.817533 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:30.817534 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:30.818020 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:30.818021 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:30.818338 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:30.818339 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:30.818388 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:30.818717 (+ 329us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:30.818720 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:30.818974 (+ 254us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:19:30.820586 (+ 1612us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:30.820586 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:30.820587 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:30.820747 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:30.820918 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:30.820918 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:30.820918 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:30.820968 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:30.821193 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:19:30.821194 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:19:30.821228 (+ 34us) client_negotiation.cc:769] Sending connection context
0812 01:19:30.821280 (+ 52us) client_negotiation.cc:240] Negotiation successful
0812 01:19:30.821353 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":77,"threads_started":1}
I20260812 01:19:30.822634 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 4718
I20260812 01:19:30.822736 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/minicluster-data/ts-1/wal/instance
I20260812 01:19:30.822901 24578 external_mini_cluster.cc:1428] Setting key 6f475cd3784c31882b72e0879efe4ca7
WARNING: no policy specified for oryx/127.24.0.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.0.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.0.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/krb5kdc/oryx_127.24.0.131.keytab.
Entry for principal oryx/127.24.0.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/krb5kdc/oryx_127.24.0.131.keytab.
I20260812 01:19:30.851576 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.0.131:0
--local_ip_for_outbound_sockets=127.24.0.131
--webserver_interface=127.24.0.131
--webserver_port=0
--tserver_master_addrs=127.24.0.190:39395
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/krb5kdc/oryx_127.24.0.131.keytab
--principal=oryx/127.24.0.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:45119
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:19:30.936559 4931 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:19:30.936779 4931 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:19:30.936820 4931 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:19:30.937944 4931 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:19:30.938017 4931 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:19:30.938055 4931 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 01:19:30.938112 4931 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.0.131
I20260812 01:19:30.939611 4931 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:45119
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/krb5kdc/oryx_127.24.0.131.keytab
--principal=oryx/127.24.0.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.0.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.0.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.0.190:39395
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4931
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.0.131
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:19:30.939872 4931 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:19:30.940148 4931 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:19:30.945283 4946 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:30.945214 4941 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:19:30.945432 4931 server_base.cc:1034] running on GCE node
W20260812 01:19:30.945472 4944 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:19:30.945798 4931 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:19:30.946089 4931 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:19:30.947268 4931 hybrid_clock.cc:648] HybridClock initialized: now 1786497570947256 us; error 29 us; skew 500 ppm
Aug 12 01:19:30 dist-test-slave-t3q3 krb5kdc[4301](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497570, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.0.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:19:30.949801 4931 init.cc:377] Logged in from keytab as oryx/127.24.0.131@KRBTEST.COM (short username oryx)
I20260812 01:19:30.950237 4931 webserver.cc:460] Webserver started at http://127.24.0.131:43497/ using document root <none> and password file <none>
I20260812 01:19:30.950470 4931 fs_manager.cc:359] Metadata directory not provided
I20260812 01:19:30.950541 4931 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:19:30.950699 4931 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:19:30.951725 4931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/minicluster-data/ts-2/data/instance:
uuid: "7107058860f74ceab1ce49475d781b6c"
format_stamp: "Formatted at 2026-08-12 01:19:30 on dist-test-slave-t3q3"
server_key: "1ccaa44c219cb8740275bd9fbe7ffc2f"
server_key_iv: "52f6df8ece4eede4b7f063ff1317e420"
server_key_version: "encryptionkey@0"
I20260812 01:19:30.952148 4931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/minicluster-data/ts-2/wal/instance:
uuid: "7107058860f74ceab1ce49475d781b6c"
format_stamp: "Formatted at 2026-08-12 01:19:30 on dist-test-slave-t3q3"
server_key: "1ccaa44c219cb8740275bd9fbe7ffc2f"
server_key_iv: "52f6df8ece4eede4b7f063ff1317e420"
server_key_version: "encryptionkey@0"
I20260812 01:19:30.953583 4931 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:19:30.954294 4956 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:30.954480 4931 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 01:19:30.954579 4931 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/minicluster-data/ts-2/wal
uuid: "7107058860f74ceab1ce49475d781b6c"
format_stamp: "Formatted at 2026-08-12 01:19:30 on dist-test-slave-t3q3"
server_key: "1ccaa44c219cb8740275bd9fbe7ffc2f"
server_key_iv: "52f6df8ece4eede4b7f063ff1317e420"
server_key_version: "encryptionkey@0"
I20260812 01:19:30.954667 4931 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:19:30.977319 4931 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:19:30.979266 4931 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:19:30.979491 4931 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:19:30.980011 4931 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:19:30.980105 4931 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:30.980150 4931 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:19:30.980185 4931 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 01:19:30 dist-test-slave-t3q3 krb5kdc[4301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497570, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.0.131@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM
I20260812 01:19:30.987751 4931 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.131:38125
I20260812 01:19:30.987874 5129 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.131:38125 every 8 connection(s)
I20260812 01:19:30.988201 4931 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/minicluster-data/ts-2/data/info.pb
I20260812 01:19:30.990370 4462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:30.980508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.131:60855 (local address 127.24.0.190:39395)
0812 01:19:30.980584 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:30.980587 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:30.980851 (+ 264us) server_negotiation.cc:407] Connection header received
0812 01:19:30.981562 (+ 711us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:30.981563 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:30.981575 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:30.981662 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:30.982595 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:30.983150 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:30.984877 (+ 1727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:30.985087 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:30.985170 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:30.987500 (+ 2330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:30.987502 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:30.987503 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:30.987511 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:30.989274 (+ 1763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:30.989627 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:30.989628 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:30.989628 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:30.989680 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:30.989855 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:30.989856 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:30.989856 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:30.989952 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:30.990002 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:30.990239 (+ 237us) server_negotiation.cc:299] Negotiation successful
0812 01:19:30.990305 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 01:19:30.990515 4968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:30.980485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:39395 (local address 127.24.0.131:60855)
0812 01:19:30.980687 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:30.980743 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:30.981451 (+ 708us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:30.981754 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:30.981759 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:30.982166 (+ 407us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:30.982474 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:30.982482 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:30.983251 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:30.983252 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:30.984762 (+ 1510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:30.984767 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:30.985209 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:30.985210 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:30.985336 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:30.985830 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:30.985841 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:30.987370 (+ 1529us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:19:30.989343 (+ 1973us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:30.989348 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:30.989349 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:30.989561 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:30.989721 (+ 160us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:30.989722 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:30.989722 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:30.989806 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:30.990010 (+ 204us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:19:30.990015 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:19:30.990125 (+ 110us) client_negotiation.cc:769] Sending connection context
0812 01:19:30.990208 (+ 83us) client_negotiation.cc:240] Negotiation successful
0812 01:19:30.990323 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":109,"threads_started":1}
I20260812 01:19:30.990990 5130 heartbeater.cc:347] Connected to a master server at 127.24.0.190:39395
I20260812 01:19:30.991079 5130 heartbeater.cc:464] Registering TS with master...
I20260812 01:19:30.991269 5130 heartbeater.cc:511] Master 127.24.0.190:39395 requested a full tablet report, sending...
I20260812 01:19:30.991691 4375 ts_manager.cc:194] Registered new tserver with Master: 7107058860f74ceab1ce49475d781b6c (127.24.0.131:38125)
I20260812 01:19:30.992161 4375 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.0.131@KRBTEST.COM'} at 127.24.0.131:60855
I20260812 01:19:30.997501 4462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:30.992135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.131:47109 (local address 127.24.0.190:39395)
0812 01:19:30.992175 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:30.992177 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:30.992308 (+ 131us) server_negotiation.cc:407] Connection header received
0812 01:19:30.992374 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:30.992375 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:30.992386 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:30.992443 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:30.992839 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:30.993346 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:30.993997 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:30.994175 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:30.994225 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:30.995036 (+ 811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:30.995037 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:30.995037 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:30.995043 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:30.996489 (+ 1446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:30.996869 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:30.996870 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:30.996870 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:30.996920 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:30.997125 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:30.997125 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:30.997126 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:30.997221 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:30.997272 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:30.997382 (+ 110us) server_negotiation.cc:299] Negotiation successful
0812 01:19:30.997439 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 01:19:30.997519 5139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:30.992031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:39395 (local address 127.24.0.131:47109)
0812 01:19:30.992243 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:30.992255 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:30.992321 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:30.992455 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:30.992456 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:30.992687 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:30.992760 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:30.992763 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:30.993440 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:30.993441 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:30.993928 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:30.993930 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:30.994239 (+ 309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:30.994239 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:30.994288 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:30.994677 (+ 389us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:30.994681 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:30.994955 (+ 274us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:19:30.996628 (+ 1673us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:30.996629 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:30.996629 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:30.996798 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:30.996980 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:30.996981 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:30.996981 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:30.997068 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:30.997276 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:19:30.997277 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:19:30.997315 (+ 38us) client_negotiation.cc:769] Sending connection context
0812 01:19:30.997368 (+ 53us) client_negotiation.cc:240] Negotiation successful
0812 01:19:30.997445 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":122,"threads_started":1}
I20260812 01:19:30.998265 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 4931
I20260812 01:19:30.998374 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/minicluster-data/ts-2/wal/instance
I20260812 01:19:30.998603 24578 external_mini_cluster.cc:1428] Setting key 36e08e660bb6925e285f97b59455d605
I20260812 01:19:30.999646 24578 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 01:19:31 dist-test-slave-t3q3 krb5kdc[4301](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497571, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 01:19:31 dist-test-slave-t3q3 krb5kdc[4301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:19:31 dist-test-slave-t3q3 krb5kdc[4301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
W20260812 01:19:31.022447 4462 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:19:31.016929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44764 (local address 127.24.0.190:39395)
0812 01:19:31.016984 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:31.016986 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:31.016995 (+ 9us) server_negotiation.cc:407] Connection header received
0812 01:19:31.017053 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:31.017054 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:31.017063 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:31.017120 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:31.017697 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:31.018186 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:31.018931 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:31.019143 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:31.019209 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:31.022323 (+ 3114us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44764: BlockingRecv error: recv got EOF from 127.0.0.1:44764 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
Aug 12 01:19:31 dist-test-slave-t3q3 krb5kdc[4301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497571, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM
I20260812 01:19:31.030299 4462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:31.023151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44772 (local address 127.24.0.190:39395)
0812 01:19:31.023229 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:31.023232 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:31.023323 (+ 91us) server_negotiation.cc:407] Connection header received
0812 01:19:31.023414 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:31.023416 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:31.023426 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:31.023482 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:31.023958 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:31.024506 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:31.025278 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:31.025481 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:31.025559 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:31.027763 (+ 2204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:31.027764 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:31.027765 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:31.027775 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:31.029155 (+ 1380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:31.029507 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:31.029509 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:31.029509 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:31.029568 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:31.029774 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:31.029775 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:31.029775 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:31.029890 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:31.030006 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:31.030113 (+ 107us) server_negotiation.cc:299] Negotiation successful
0812 01:19:31.030190 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 01:19:31.032332 4375 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44772:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 01:19:31.034009 4375 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 01:19:31.040805 5163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:31.036480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.131:38125 (local address 127.0.0.1:34842)
0812 01:19:31.036684 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:31.036748 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:31.036819 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:31.037303 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:31.037304 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:31.037307 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:19:31.037371 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:31.037374 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:31.038282 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:31.038283 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:31.039518 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:31.039520 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:31.040490 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:31.040491 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:31.040578 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:31.040593 (+ 15us) client_negotiation.cc:769] Sending connection context
0812 01:19:31.040670 (+ 77us) client_negotiation.cc:240] Negotiation successful
0812 01:19:31.040731 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":61,"threads_started":1}
I20260812 01:19:31.040928 5162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:31.036721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.130:39435 (local address 127.0.0.1:40224)
0812 01:19:31.037125 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:31.037135 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:31.037185 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:31.037381 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:31.037382 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:31.037383 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:19:31.037426 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:31.037427 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:31.038671 (+ 1244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:31.038673 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:31.039715 (+ 1042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:31.039717 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:31.040664 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:31.040665 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:31.040734 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:31.040738 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 01:19:31.040802 (+ 64us) client_negotiation.cc:240] Negotiation successful
0812 01:19:31.040847 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":70,"threads_started":1}
I20260812 01:19:31.040946 5170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:31.036851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40224 (local address 127.24.0.130:39435)
0812 01:19:31.037065 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:31.037069 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:31.037240 (+ 171us) server_negotiation.cc:407] Connection header received
0812 01:19:31.037280 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:31.037282 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:31.037296 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:31.037373 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:19:31.037492 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:31.038584 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:31.039795 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:31.040562 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:31.040663 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:31.040743 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:31.040817 (+ 74us) server_negotiation.cc:299] Negotiation successful
0812 01:19:31.040855 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":104,"threads_started":1}
I20260812 01:19:31.041465 5164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:31.036619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34842 (local address 127.24.0.131:38125)
0812 01:19:31.037103 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:31.037108 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:31.037118 (+ 10us) server_negotiation.cc:407] Connection header received
0812 01:19:31.037175 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:31.037177 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:31.037195 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:31.037451 (+ 256us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:19:31.037510 (+ 59us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:31.038190 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:31.039605 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:31.040386 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:31.040462 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:31.040541 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:31.041316 (+ 775us) server_negotiation.cc:299] Negotiation successful
0812 01:19:31.041362 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":447,"thread_start_us":88,"threads_started":1}
I20260812 01:19:31.041689 4820 tablet_service.cc:1467] Processing CreateTablet for tablet 1401049ef7e2461e8491c205a2d505a5 (DEFAULT_TABLE table=test-table [id=e8c44c2f0bcd4051930a3edd7154dfb7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:19:31.042027 5167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:31.036488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.129:33711 (local address 127.0.0.1:55444)
0812 01:19:31.036935 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:31.036944 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:31.036982 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:31.037260 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:31.037263 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:31.037265 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:19:31.037345 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:31.037347 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:31.039786 (+ 2439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:31.039786 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:31.040674 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:31.040677 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:31.041800 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:31.041801 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:31.041854 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:31.041857 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 01:19:31.041932 (+ 75us) client_negotiation.cc:240] Negotiation successful
0812 01:19:31.041974 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":118,"threads_started":1}
I20260812 01:19:31.042096 4820 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1401049ef7e2461e8491c205a2d505a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:19:31.042083 5036 tablet_service.cc:1467] Processing CreateTablet for tablet 1401049ef7e2461e8491c205a2d505a5 (DEFAULT_TABLE table=test-table [id=e8c44c2f0bcd4051930a3edd7154dfb7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:19:31.042305 5168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:31.036832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55444 (local address 127.24.0.129:33711)
0812 01:19:31.036996 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:31.037002 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:31.037014 (+ 12us) server_negotiation.cc:407] Connection header received
0812 01:19:31.037057 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:31.037059 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:31.037076 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:31.037170 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:19:31.037430 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:31.038567 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:31.040769 (+ 2202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:31.041609 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:31.041681 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:31.041768 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:31.042152 (+ 384us) server_negotiation.cc:299] Negotiation successful
0812 01:19:31.042203 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":87,"threads_started":1}
I20260812 01:19:31.042421 5036 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1401049ef7e2461e8491c205a2d505a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:19:31.044812 4605 tablet_service.cc:1467] Processing CreateTablet for tablet 1401049ef7e2461e8491c205a2d505a5 (DEFAULT_TABLE table=test-table [id=e8c44c2f0bcd4051930a3edd7154dfb7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:19:31.045164 4605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1401049ef7e2461e8491c205a2d505a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:19:31.046200 5172 tablet_bootstrap.cc:492] T 1401049ef7e2461e8491c205a2d505a5 P 7107058860f74ceab1ce49475d781b6c: Bootstrap starting.
I20260812 01:19:31.046360 5173 tablet_bootstrap.cc:492] T 1401049ef7e2461e8491c205a2d505a5 P 9b55292de1ce492b9e347c66f4cf0111: Bootstrap starting.
I20260812 01:19:31.046939 5173 tablet_bootstrap.cc:654] T 1401049ef7e2461e8491c205a2d505a5 P 9b55292de1ce492b9e347c66f4cf0111: Neither blocks nor log segments found. Creating new log.
I20260812 01:19:31.047060 5172 tablet_bootstrap.cc:654] T 1401049ef7e2461e8491c205a2d505a5 P 7107058860f74ceab1ce49475d781b6c: Neither blocks nor log segments found. Creating new log.
I20260812 01:19:31.047312 5173 log.cc:824] T 1401049ef7e2461e8491c205a2d505a5 P 9b55292de1ce492b9e347c66f4cf0111: Log is configured to *not* fsync() on all Append() calls
I20260812 01:19:31.047482 5172 log.cc:824] T 1401049ef7e2461e8491c205a2d505a5 P 7107058860f74ceab1ce49475d781b6c: Log is configured to *not* fsync() on all Append() calls
I20260812 01:19:31.048266 5173 tablet_bootstrap.cc:492] T 1401049ef7e2461e8491c205a2d505a5 P 9b55292de1ce492b9e347c66f4cf0111: No bootstrap required, opened a new log
I20260812 01:19:31.048342 5173 ts_tablet_manager.cc:1397] T 1401049ef7e2461e8491c205a2d505a5 P 9b55292de1ce492b9e347c66f4cf0111: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 01:19:31.048358 5172 tablet_bootstrap.cc:492] T 1401049ef7e2461e8491c205a2d505a5 P 7107058860f74ceab1ce49475d781b6c: No bootstrap required, opened a new log
I20260812 01:19:31.048422 5172 ts_tablet_manager.cc:1397] T 1401049ef7e2461e8491c205a2d505a5 P 7107058860f74ceab1ce49475d781b6c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 01:19:31.049151 5179 tablet_bootstrap.cc:492] T 1401049ef7e2461e8491c205a2d505a5 P 0ffa5375dd2b431b852b9dd97c25d204: Bootstrap starting.
I20260812 01:19:31.049841 5179 tablet_bootstrap.cc:654] T 1401049ef7e2461e8491c205a2d505a5 P 0ffa5375dd2b431b852b9dd97c25d204: Neither blocks nor log segments found. Creating new log.
I20260812 01:19:31.050170 5179 log.cc:824] T 1401049ef7e2461e8491c205a2d505a5 P 0ffa5375dd2b431b852b9dd97c25d204: Log is configured to *not* fsync() on all Append() calls
I20260812 01:19:31.050104 5173 raft_consensus.cc:348] T 1401049ef7e2461e8491c205a2d505a5 P 9b55292de1ce492b9e347c66f4cf0111 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7107058860f74ceab1ce49475d781b6c" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 38125 } } peers { permanent_uuid: "0ffa5375dd2b431b852b9dd97c25d204" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 33711 } } peers { permanent_uuid: "9b55292de1ce492b9e347c66f4cf0111" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39435 } }
I20260812 01:19:31.050258 5173 raft_consensus.cc:374] T 1401049ef7e2461e8491c205a2d505a5 P 9b55292de1ce492b9e347c66f4cf0111 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:19:31.050302 5173 raft_consensus.cc:729] T 1401049ef7e2461e8491c205a2d505a5 P 9b55292de1ce492b9e347c66f4cf0111 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b55292de1ce492b9e347c66f4cf0111, State: Initialized, Role: FOLLOWER
I20260812 01:19:31.050417 5173 consensus_queue.cc:260] T 1401049ef7e2461e8491c205a2d505a5 P 9b55292de1ce492b9e347c66f4cf0111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7107058860f74ceab1ce49475d781b6c" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 38125 } } peers { permanent_uuid: "0ffa5375dd2b431b852b9dd97c25d204" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 33711 } } peers { permanent_uuid: "9b55292de1ce492b9e347c66f4cf0111" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39435 } }
I20260812 01:19:31.050701 5173 ts_tablet_manager.cc:1428] T 1401049ef7e2461e8491c205a2d505a5 P 9b55292de1ce492b9e347c66f4cf0111: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 01:19:31.050421 5172 raft_consensus.cc:348] T 1401049ef7e2461e8491c205a2d505a5 P 7107058860f74ceab1ce49475d781b6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7107058860f74ceab1ce49475d781b6c" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 38125 } } peers { permanent_uuid: "0ffa5375dd2b431b852b9dd97c25d204" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 33711 } } peers { permanent_uuid: "9b55292de1ce492b9e347c66f4cf0111" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39435 } }
I20260812 01:19:31.050838 5172 raft_consensus.cc:374] T 1401049ef7e2461e8491c205a2d505a5 P 7107058860f74ceab1ce49475d781b6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:19:31.050881 5172 raft_consensus.cc:729] T 1401049ef7e2461e8491c205a2d505a5 P 7107058860f74ceab1ce49475d781b6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7107058860f74ceab1ce49475d781b6c, State: Initialized, Role: FOLLOWER
I20260812 01:19:31.050951 5179 tablet_bootstrap.cc:492] T 1401049ef7e2461e8491c205a2d505a5 P 0ffa5375dd2b431b852b9dd97c25d204: No bootstrap required, opened a new log
I20260812 01:19:31.050974 5172 consensus_queue.cc:260] T 1401049ef7e2461e8491c205a2d505a5 P 7107058860f74ceab1ce49475d781b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7107058860f74ceab1ce49475d781b6c" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 38125 } } peers { permanent_uuid: "0ffa5375dd2b431b852b9dd97c25d204" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 33711 } } peers { permanent_uuid: "9b55292de1ce492b9e347c66f4cf0111" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39435 } }
I20260812 01:19:31.051033 5179 ts_tablet_manager.cc:1397] T 1401049ef7e2461e8491c205a2d505a5 P 0ffa5375dd2b431b852b9dd97c25d204: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 01:19:31.051221 5172 ts_tablet_manager.cc:1428] T 1401049ef7e2461e8491c205a2d505a5 P 7107058860f74ceab1ce49475d781b6c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 01:19:31.051241 5130 heartbeater.cc:503] Master 127.24.0.190:39395 was elected leader, sending a full tablet report...
I20260812 01:19:31.052150 4921 heartbeater.cc:503] Master 127.24.0.190:39395 was elected leader, sending a full tablet report...
I20260812 01:19:31.052886 5179 raft_consensus.cc:348] T 1401049ef7e2461e8491c205a2d505a5 P 0ffa5375dd2b431b852b9dd97c25d204 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7107058860f74ceab1ce49475d781b6c" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 38125 } } peers { permanent_uuid: "0ffa5375dd2b431b852b9dd97c25d204" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 33711 } } peers { permanent_uuid: "9b55292de1ce492b9e347c66f4cf0111" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39435 } }
I20260812 01:19:31.053020 5179 raft_consensus.cc:374] T 1401049ef7e2461e8491c205a2d505a5 P 0ffa5375dd2b431b852b9dd97c25d204 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:19:31.053041 5179 raft_consensus.cc:729] T 1401049ef7e2461e8491c205a2d505a5 P 0ffa5375dd2b431b852b9dd97c25d204 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ffa5375dd2b431b852b9dd97c25d204, State: Initialized, Role: FOLLOWER
I20260812 01:19:31.053174 5179 consensus_queue.cc:260] T 1401049ef7e2461e8491c205a2d505a5 P 0ffa5375dd2b431b852b9dd97c25d204 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7107058860f74ceab1ce49475d781b6c" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 38125 } } peers { permanent_uuid: "0ffa5375dd2b431b852b9dd97c25d204" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 33711 } } peers { permanent_uuid: "9b55292de1ce492b9e347c66f4cf0111" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39435 } }
I20260812 01:19:31.053459 5179 ts_tablet_manager.cc:1428] T 1401049ef7e2461e8491c205a2d505a5 P 0ffa5375dd2b431b852b9dd97c25d204: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260812 01:19:31.053478 4703 heartbeater.cc:503] Master 127.24.0.190:39395 was elected leader, sending a full tablet report...
W20260812 01:19:31.063057 4922 tablet.cc:2369] T 1401049ef7e2461e8491c205a2d505a5 P 9b55292de1ce492b9e347c66f4cf0111: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 01:19:31.143778 4706 tablet.cc:2369] T 1401049ef7e2461e8491c205a2d505a5 P 0ffa5375dd2b431b852b9dd97c25d204: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 01:19:31.238880 5131 tablet.cc:2369] T 1401049ef7e2461e8491c205a2d505a5 P 7107058860f74ceab1ce49475d781b6c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 01:19:31.361441 5182 raft_consensus.cc:482] T 1401049ef7e2461e8491c205a2d505a5 P 9b55292de1ce492b9e347c66f4cf0111 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 01:19:31.361617 5182 raft_consensus.cc:504] T 1401049ef7e2461e8491c205a2d505a5 P 9b55292de1ce492b9e347c66f4cf0111 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7107058860f74ceab1ce49475d781b6c" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 38125 } } peers { permanent_uuid: "0ffa5375dd2b431b852b9dd97c25d204" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 33711 } } peers { permanent_uuid: "9b55292de1ce492b9e347c66f4cf0111" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39435 } }
I20260812 01:19:31.362020 5182 leader_election.cc:288] T 1401049ef7e2461e8491c205a2d505a5 P 9b55292de1ce492b9e347c66f4cf0111 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7107058860f74ceab1ce49475d781b6c (127.24.0.131:38125), 0ffa5375dd2b431b852b9dd97c25d204 (127.24.0.129:33711)
I20260812 01:19:31.365121 5164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:31.362361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.130:46999 (local address 127.24.0.131:38125)
0812 01:19:31.362411 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:31.362413 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:31.362531 (+ 118us) server_negotiation.cc:407] Connection header received
0812 01:19:31.362614 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:31.362615 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:31.362627 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:31.362699 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:19:31.362888 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:31.363356 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:31.364408 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:31.364770 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:31.364859 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:31.364869 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:31.365012 (+ 143us) server_negotiation.cc:299] Negotiation successful
0812 01:19:31.365055 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 01:19:31.365149 5187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:31.362258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.131:38125 (local address 127.24.0.130:46999)
0812 01:19:31.362437 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:31.362448 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:31.362550 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:31.362716 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:31.362717 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:31.362719 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:19:31.362811 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:31.362814 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:31.363438 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:31.363440 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:31.364316 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:31.364318 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:31.364859 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:31.364860 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:31.364936 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:31.364939 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 01:19:31.365006 (+ 67us) client_negotiation.cc:240] Negotiation successful
0812 01:19:31.365049 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"mutex_wait_us":20,"thread_start_us":66,"threads_started":1}
I20260812 01:19:31.365229 5168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:31.362347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.130:51469 (local address 127.24.0.129:33711)
0812 01:19:31.362402 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:31.362405 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:31.362484 (+ 79us) server_negotiation.cc:407] Connection header received
0812 01:19:31.362596 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:31.362597 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:31.362616 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:31.362704 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:19:31.362888 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:31.363356 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:31.364504 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:31.364861 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:31.364952 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:31.364960 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:31.365107 (+ 147us) server_negotiation.cc:299] Negotiation successful
0812 01:19:31.365145 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 01:19:31.365473 5186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:31.362258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.129:33711 (local address 127.24.0.130:51469)
0812 01:19:31.362397 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:31.362410 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:31.362504 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:31.362715 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:31.362716 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:31.362718 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:19:31.362815 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:31.362816 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:31.363445 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:31.363446 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:31.364412 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:31.364415 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:31.364952 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:31.364953 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:31.365030 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:31.365034 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 01:19:31.365102 (+ 68us) client_negotiation.cc:240] Negotiation successful
0812 01:19:31.365143 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"mutex_wait_us":46,"thread_start_us":67,"threads_started":1}
I20260812 01:19:31.365674 5069 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1401049ef7e2461e8491c205a2d505a5" candidate_uuid: "9b55292de1ce492b9e347c66f4cf0111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7107058860f74ceab1ce49475d781b6c" is_pre_election: true
I20260812 01:19:31.365852 5069 raft_consensus.cc:2454] T 1401049ef7e2461e8491c205a2d505a5 P 7107058860f74ceab1ce49475d781b6c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9b55292de1ce492b9e347c66f4cf0111 in term 0.
I20260812 01:19:31.365834 4636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1401049ef7e2461e8491c205a2d505a5" candidate_uuid: "9b55292de1ce492b9e347c66f4cf0111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ffa5375dd2b431b852b9dd97c25d204" is_pre_election: true
I20260812 01:19:31.365975 4636 raft_consensus.cc:2454] T 1401049ef7e2461e8491c205a2d505a5 P 0ffa5375dd2b431b852b9dd97c25d204 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9b55292de1ce492b9e347c66f4cf0111 in term 0.
I20260812 01:19:31.366114 4748 leader_election.cc:302] T 1401049ef7e2461e8491c205a2d505a5 P 9b55292de1ce492b9e347c66f4cf0111 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7107058860f74ceab1ce49475d781b6c, 9b55292de1ce492b9e347c66f4cf0111; no voters:
I20260812 01:19:31.366315 5182 raft_consensus.cc:2781] T 1401049ef7e2461e8491c205a2d505a5 P 9b55292de1ce492b9e347c66f4cf0111 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 01:19:31.366374 5182 raft_consensus.cc:482] T 1401049ef7e2461e8491c205a2d505a5 P 9b55292de1ce492b9e347c66f4cf0111 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 01:19:31.366405 5182 raft_consensus.cc:3037] T 1401049ef7e2461e8491c205a2d505a5 P 9b55292de1ce492b9e347c66f4cf0111 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:19:31.367192 5182 raft_consensus.cc:504] T 1401049ef7e2461e8491c205a2d505a5 P 9b55292de1ce492b9e347c66f4cf0111 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7107058860f74ceab1ce49475d781b6c" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 38125 } } peers { permanent_uuid: "0ffa5375dd2b431b852b9dd97c25d204" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 33711 } } peers { permanent_uuid: "9b55292de1ce492b9e347c66f4cf0111" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39435 } }
I20260812 01:19:31.367355 5182 leader_election.cc:288] T 1401049ef7e2461e8491c205a2d505a5 P 9b55292de1ce492b9e347c66f4cf0111 [CANDIDATE]: Term 1 election: Requested vote from peers 7107058860f74ceab1ce49475d781b6c (127.24.0.131:38125), 0ffa5375dd2b431b852b9dd97c25d204 (127.24.0.129:33711)
I20260812 01:19:31.367537 5069 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1401049ef7e2461e8491c205a2d505a5" candidate_uuid: "9b55292de1ce492b9e347c66f4cf0111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7107058860f74ceab1ce49475d781b6c"
I20260812 01:19:31.367606 4636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1401049ef7e2461e8491c205a2d505a5" candidate_uuid: "9b55292de1ce492b9e347c66f4cf0111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ffa5375dd2b431b852b9dd97c25d204"
I20260812 01:19:31.367645 5069 raft_consensus.cc:3037] T 1401049ef7e2461e8491c205a2d505a5 P 7107058860f74ceab1ce49475d781b6c [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:19:31.367681 4636 raft_consensus.cc:3037] T 1401049ef7e2461e8491c205a2d505a5 P 0ffa5375dd2b431b852b9dd97c25d204 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:19:31.368559 5069 raft_consensus.cc:2454] T 1401049ef7e2461e8491c205a2d505a5 P 7107058860f74ceab1ce49475d781b6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b55292de1ce492b9e347c66f4cf0111 in term 1.
I20260812 01:19:31.368559 4636 raft_consensus.cc:2454] T 1401049ef7e2461e8491c205a2d505a5 P 0ffa5375dd2b431b852b9dd97c25d204 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b55292de1ce492b9e347c66f4cf0111 in term 1.
I20260812 01:19:31.368762 4748 leader_election.cc:302] T 1401049ef7e2461e8491c205a2d505a5 P 9b55292de1ce492b9e347c66f4cf0111 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7107058860f74ceab1ce49475d781b6c, 9b55292de1ce492b9e347c66f4cf0111; no voters:
I20260812 01:19:31.368863 5182 raft_consensus.cc:2781] T 1401049ef7e2461e8491c205a2d505a5 P 9b55292de1ce492b9e347c66f4cf0111 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:19:31.368978 5182 raft_consensus.cc:686] T 1401049ef7e2461e8491c205a2d505a5 P 9b55292de1ce492b9e347c66f4cf0111 [term 1 LEADER]: Becoming Leader. State: Replica: 9b55292de1ce492b9e347c66f4cf0111, State: Running, Role: LEADER
I20260812 01:19:31.369077 5182 consensus_queue.cc:237] T 1401049ef7e2461e8491c205a2d505a5 P 9b55292de1ce492b9e347c66f4cf0111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7107058860f74ceab1ce49475d781b6c" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 38125 } } peers { permanent_uuid: "0ffa5375dd2b431b852b9dd97c25d204" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 33711 } } peers { permanent_uuid: "9b55292de1ce492b9e347c66f4cf0111" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39435 } }
I20260812 01:19:31.369935 4375 catalog_manager.cc:5495] T 1401049ef7e2461e8491c205a2d505a5 P 9b55292de1ce492b9e347c66f4cf0111 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9b55292de1ce492b9e347c66f4cf0111 (127.24.0.130). New cstate: current_term: 1 leader_uuid: "9b55292de1ce492b9e347c66f4cf0111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7107058860f74ceab1ce49475d781b6c" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 38125 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ffa5375dd2b431b852b9dd97c25d204" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 33711 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9b55292de1ce492b9e347c66f4cf0111" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39435 } health_report { overall_health: HEALTHY } } }
I20260812 01:19:31.395511 5170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:31.393391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40240 (local address 127.24.0.130:39435)
0812 01:19:31.393456 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:31.393458 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:31.393466 (+ 8us) server_negotiation.cc:407] Connection header received
0812 01:19:31.393501 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:31.393502 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:31.393523 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:31.393587 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:19:31.393689 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:31.394163 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:31.394862 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:31.395040 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:31.395104 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:31.395217 (+ 113us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:19:31.395309 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:19:31.395363 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:31.395397 (+ 34us) server_negotiation.cc:299] Negotiation successful
0812 01:19:31.395413 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 01:19:31.397439 5069 raft_consensus.cc:1261] T 1401049ef7e2461e8491c205a2d505a5 P 7107058860f74ceab1ce49475d781b6c [term 1 FOLLOWER]: Refusing update from remote peer 9b55292de1ce492b9e347c66f4cf0111: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:19:31.397439 4636 raft_consensus.cc:1261] T 1401049ef7e2461e8491c205a2d505a5 P 0ffa5375dd2b431b852b9dd97c25d204 [term 1 FOLLOWER]: Refusing update from remote peer 9b55292de1ce492b9e347c66f4cf0111: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:19:31.397828 5189 consensus_queue.cc:1035] T 1401049ef7e2461e8491c205a2d505a5 P 9b55292de1ce492b9e347c66f4cf0111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7107058860f74ceab1ce49475d781b6c" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 38125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:19:31.397989 5182 consensus_queue.cc:1035] T 1401049ef7e2461e8491c205a2d505a5 P 9b55292de1ce492b9e347c66f4cf0111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ffa5375dd2b431b852b9dd97c25d204" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 33711 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:19:31.400228 5194 mvcc.cc:205] Tried to move back new op lower bound from 7317494052440891392 to 7317494052328452096. Current Snapshot: MvccSnapshot[applied={T|T < 7317494052440891392}]
I20260812 01:19:31.402887 4372 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44772:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:19:31.402985 4372 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44772:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:19:31.403810 4372 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8cf578e52d544713ab6c35cb1333039f: Sending DeleteTablet for 3 replicas of tablet 1401049ef7e2461e8491c205a2d505a5
I20260812 01:19:31.404215 5036 tablet_service.cc:1514] Processing DeleteTablet for tablet 1401049ef7e2461e8491c205a2d505a5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:19:31 UTC) from {username='oryx', principal='oryx/127.24.0.190@KRBTEST.COM'} at 127.0.0.1:34842
I20260812 01:19:31.404316 4820 tablet_service.cc:1514] Processing DeleteTablet for tablet 1401049ef7e2461e8491c205a2d505a5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:19:31 UTC) from {username='oryx', principal='oryx/127.24.0.190@KRBTEST.COM'} at 127.0.0.1:40224
I20260812 01:19:31.404455 4605 tablet_service.cc:1514] Processing DeleteTablet for tablet 1401049ef7e2461e8491c205a2d505a5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:19:31 UTC) from {username='oryx', principal='oryx/127.24.0.190@KRBTEST.COM'} at 127.0.0.1:55444
I20260812 01:19:31.404536 5220 tablet_replica.cc:330] T 1401049ef7e2461e8491c205a2d505a5 P 9b55292de1ce492b9e347c66f4cf0111: stopping tablet replica
I20260812 01:19:31.404625 5220 raft_consensus.cc:2229] T 1401049ef7e2461e8491c205a2d505a5 P 9b55292de1ce492b9e347c66f4cf0111 [term 1 LEADER]: Raft consensus shutting down.
I20260812 01:19:31.404654 4372 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44772:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260812 01:19:31.404748 5220 raft_consensus.cc:2258] T 1401049ef7e2461e8491c205a2d505a5 P 9b55292de1ce492b9e347c66f4cf0111 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260812 01:19:31.404798 4372 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 01:19:31.404891 5218 tablet_replica.cc:330] T 1401049ef7e2461e8491c205a2d505a5 P 7107058860f74ceab1ce49475d781b6c: stopping tablet replica
I20260812 01:19:31.404986 5218 raft_consensus.cc:2229] T 1401049ef7e2461e8491c205a2d505a5 P 7107058860f74ceab1ce49475d781b6c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 01:19:31.405392 5218 raft_consensus.cc:2258] T 1401049ef7e2461e8491c205a2d505a5 P 7107058860f74ceab1ce49475d781b6c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:19:31.405880 5220 ts_tablet_manager.cc:1905] T 1401049ef7e2461e8491c205a2d505a5 P 9b55292de1ce492b9e347c66f4cf0111: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:19:31.405248 5221 tablet_replica.cc:330] T 1401049ef7e2461e8491c205a2d505a5 P 0ffa5375dd2b431b852b9dd97c25d204: stopping tablet replica
I20260812 01:19:31.406045 5221 raft_consensus.cc:2229] T 1401049ef7e2461e8491c205a2d505a5 P 0ffa5375dd2b431b852b9dd97c25d204 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 01:19:31.406143 5221 raft_consensus.cc:2258] T 1401049ef7e2461e8491c205a2d505a5 P 0ffa5375dd2b431b852b9dd97c25d204 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:19:31.406137 5036 tablet_service.cc:1467] Processing CreateTablet for tablet d6a604a4264d4eef992e88f8c2cb2cae (DEFAULT_TABLE table=test-table [id=12faae1386c84bc9af3c5e1c32389747]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:19:31.406260 5036 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6a604a4264d4eef992e88f8c2cb2cae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:19:31.407547 5220 ts_tablet_manager.cc:1918] T 1401049ef7e2461e8491c205a2d505a5 P 9b55292de1ce492b9e347c66f4cf0111: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 01:19:31.407618 5220 log.cc:1196] T 1401049ef7e2461e8491c205a2d505a5 P 9b55292de1ce492b9e347c66f4cf0111: Deleting WAL directory at /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/minicluster-data/ts-1/wal/wals/1401049ef7e2461e8491c205a2d505a5
I20260812 01:19:31.407611 5172 tablet_bootstrap.cc:492] T d6a604a4264d4eef992e88f8c2cb2cae P 7107058860f74ceab1ce49475d781b6c: Bootstrap starting.
I20260812 01:19:31.407909 5220 ts_tablet_manager.cc:1939] T 1401049ef7e2461e8491c205a2d505a5 P 9b55292de1ce492b9e347c66f4cf0111: Deleting consensus metadata
I20260812 01:19:31.407900 4605 tablet_service.cc:1467] Processing CreateTablet for tablet d6a604a4264d4eef992e88f8c2cb2cae (DEFAULT_TABLE table=test-table [id=12faae1386c84bc9af3c5e1c32389747]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:19:31.408017 4605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6a604a4264d4eef992e88f8c2cb2cae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:19:31.408084 5172 tablet_bootstrap.cc:654] T d6a604a4264d4eef992e88f8c2cb2cae P 7107058860f74ceab1ce49475d781b6c: Neither blocks nor log segments found. Creating new log.
I20260812 01:19:31.408341 4355 catalog_manager.cc:4841] TS 9b55292de1ce492b9e347c66f4cf0111 (127.24.0.130:39435): tablet 1401049ef7e2461e8491c205a2d505a5 (table test-table [id=e8c44c2f0bcd4051930a3edd7154dfb7]) successfully deleted
I20260812 01:19:31.408432 4820 tablet_service.cc:1467] Processing CreateTablet for tablet d6a604a4264d4eef992e88f8c2cb2cae (DEFAULT_TABLE table=test-table [id=12faae1386c84bc9af3c5e1c32389747]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:19:31.408527 4820 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6a604a4264d4eef992e88f8c2cb2cae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:19:31.409348 5173 tablet_bootstrap.cc:492] T d6a604a4264d4eef992e88f8c2cb2cae P 9b55292de1ce492b9e347c66f4cf0111: Bootstrap starting.
I20260812 01:19:31.409436 5179 tablet_bootstrap.cc:492] T d6a604a4264d4eef992e88f8c2cb2cae P 0ffa5375dd2b431b852b9dd97c25d204: Bootstrap starting.
I20260812 01:19:31.409624 5218 ts_tablet_manager.cc:1905] T 1401049ef7e2461e8491c205a2d505a5 P 7107058860f74ceab1ce49475d781b6c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:19:31.409770 5173 tablet_bootstrap.cc:654] T d6a604a4264d4eef992e88f8c2cb2cae P 9b55292de1ce492b9e347c66f4cf0111: Neither blocks nor log segments found. Creating new log.
I20260812 01:19:31.409832 5179 tablet_bootstrap.cc:654] T d6a604a4264d4eef992e88f8c2cb2cae P 0ffa5375dd2b431b852b9dd97c25d204: Neither blocks nor log segments found. Creating new log.
I20260812 01:19:31.410264 5221 ts_tablet_manager.cc:1905] T 1401049ef7e2461e8491c205a2d505a5 P 0ffa5375dd2b431b852b9dd97c25d204: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:19:31.410542 5179 tablet_bootstrap.cc:492] T d6a604a4264d4eef992e88f8c2cb2cae P 0ffa5375dd2b431b852b9dd97c25d204: No bootstrap required, opened a new log
I20260812 01:19:31.410590 5179 ts_tablet_manager.cc:1397] T d6a604a4264d4eef992e88f8c2cb2cae P 0ffa5375dd2b431b852b9dd97c25d204: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260812 01:19:31.410727 5179 raft_consensus.cc:348] T d6a604a4264d4eef992e88f8c2cb2cae P 0ffa5375dd2b431b852b9dd97c25d204 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ffa5375dd2b431b852b9dd97c25d204" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 33711 } } peers { permanent_uuid: "9b55292de1ce492b9e347c66f4cf0111" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39435 } } peers { permanent_uuid: "7107058860f74ceab1ce49475d781b6c" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 38125 } }
I20260812 01:19:31.410789 5179 raft_consensus.cc:374] T d6a604a4264d4eef992e88f8c2cb2cae P 0ffa5375dd2b431b852b9dd97c25d204 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:19:31.410812 5179 raft_consensus.cc:729] T d6a604a4264d4eef992e88f8c2cb2cae P 0ffa5375dd2b431b852b9dd97c25d204 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ffa5375dd2b431b852b9dd97c25d204, State: Initialized, Role: FOLLOWER
I20260812 01:19:31.410856 5179 consensus_queue.cc:260] T d6a604a4264d4eef992e88f8c2cb2cae P 0ffa5375dd2b431b852b9dd97c25d204 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ffa5375dd2b431b852b9dd97c25d204" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 33711 } } peers { permanent_uuid: "9b55292de1ce492b9e347c66f4cf0111" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39435 } } peers { permanent_uuid: "7107058860f74ceab1ce49475d781b6c" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 38125 } }
I20260812 01:19:31.410956 5179 ts_tablet_manager.cc:1428] T d6a604a4264d4eef992e88f8c2cb2cae P 0ffa5375dd2b431b852b9dd97c25d204: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:31.411155 5218 ts_tablet_manager.cc:1918] T 1401049ef7e2461e8491c205a2d505a5 P 7107058860f74ceab1ce49475d781b6c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 01:19:31.411237 5218 log.cc:1196] T 1401049ef7e2461e8491c205a2d505a5 P 7107058860f74ceab1ce49475d781b6c: Deleting WAL directory at /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/minicluster-data/ts-2/wal/wals/1401049ef7e2461e8491c205a2d505a5
I20260812 01:19:31.411512 5218 ts_tablet_manager.cc:1939] T 1401049ef7e2461e8491c205a2d505a5 P 7107058860f74ceab1ce49475d781b6c: Deleting consensus metadata
I20260812 01:19:31.411935 4356 catalog_manager.cc:4841] TS 7107058860f74ceab1ce49475d781b6c (127.24.0.131:38125): tablet 1401049ef7e2461e8491c205a2d505a5 (table test-table [id=e8c44c2f0bcd4051930a3edd7154dfb7]) successfully deleted
I20260812 01:19:31.412014 5221 ts_tablet_manager.cc:1918] T 1401049ef7e2461e8491c205a2d505a5 P 0ffa5375dd2b431b852b9dd97c25d204: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 01:19:31.412112 5221 log.cc:1196] T 1401049ef7e2461e8491c205a2d505a5 P 0ffa5375dd2b431b852b9dd97c25d204: Deleting WAL directory at /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/minicluster-data/ts-0/wal/wals/1401049ef7e2461e8491c205a2d505a5
I20260812 01:19:31.412206 5172 tablet_bootstrap.cc:492] T d6a604a4264d4eef992e88f8c2cb2cae P 7107058860f74ceab1ce49475d781b6c: No bootstrap required, opened a new log
I20260812 01:19:31.412257 5172 ts_tablet_manager.cc:1397] T d6a604a4264d4eef992e88f8c2cb2cae P 7107058860f74ceab1ce49475d781b6c: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260812 01:19:31.412417 5172 raft_consensus.cc:348] T d6a604a4264d4eef992e88f8c2cb2cae P 7107058860f74ceab1ce49475d781b6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ffa5375dd2b431b852b9dd97c25d204" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 33711 } } peers { permanent_uuid: "9b55292de1ce492b9e347c66f4cf0111" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39435 } } peers { permanent_uuid: "7107058860f74ceab1ce49475d781b6c" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 38125 } }
I20260812 01:19:31.412495 5172 raft_consensus.cc:374] T d6a604a4264d4eef992e88f8c2cb2cae P 7107058860f74ceab1ce49475d781b6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:19:31.412541 5172 raft_consensus.cc:729] T d6a604a4264d4eef992e88f8c2cb2cae P 7107058860f74ceab1ce49475d781b6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7107058860f74ceab1ce49475d781b6c, State: Initialized, Role: FOLLOWER
I20260812 01:19:31.412583 5172 consensus_queue.cc:260] T d6a604a4264d4eef992e88f8c2cb2cae P 7107058860f74ceab1ce49475d781b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ffa5375dd2b431b852b9dd97c25d204" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 33711 } } peers { permanent_uuid: "9b55292de1ce492b9e347c66f4cf0111" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39435 } } peers { permanent_uuid: "7107058860f74ceab1ce49475d781b6c" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 38125 } }
I20260812 01:19:31.412685 5172 ts_tablet_manager.cc:1428] T d6a604a4264d4eef992e88f8c2cb2cae P 7107058860f74ceab1ce49475d781b6c: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 01:19:31.413610 5173 tablet_bootstrap.cc:492] T d6a604a4264d4eef992e88f8c2cb2cae P 9b55292de1ce492b9e347c66f4cf0111: No bootstrap required, opened a new log
I20260812 01:19:31.413652 5173 ts_tablet_manager.cc:1397] T d6a604a4264d4eef992e88f8c2cb2cae P 9b55292de1ce492b9e347c66f4cf0111: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260812 01:19:31.413755 5173 raft_consensus.cc:348] T d6a604a4264d4eef992e88f8c2cb2cae P 9b55292de1ce492b9e347c66f4cf0111 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ffa5375dd2b431b852b9dd97c25d204" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 33711 } } peers { permanent_uuid: "9b55292de1ce492b9e347c66f4cf0111" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39435 } } peers { permanent_uuid: "7107058860f74ceab1ce49475d781b6c" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 38125 } }
I20260812 01:19:31.413798 5173 raft_consensus.cc:374] T d6a604a4264d4eef992e88f8c2cb2cae P 9b55292de1ce492b9e347c66f4cf0111 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:19:31.413811 5173 raft_consensus.cc:729] T d6a604a4264d4eef992e88f8c2cb2cae P 9b55292de1ce492b9e347c66f4cf0111 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b55292de1ce492b9e347c66f4cf0111, State: Initialized, Role: FOLLOWER
I20260812 01:19:31.413872 5173 consensus_queue.cc:260] T d6a604a4264d4eef992e88f8c2cb2cae P 9b55292de1ce492b9e347c66f4cf0111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ffa5375dd2b431b852b9dd97c25d204" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 33711 } } peers { permanent_uuid: "9b55292de1ce492b9e347c66f4cf0111" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39435 } } peers { permanent_uuid: "7107058860f74ceab1ce49475d781b6c" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 38125 } }
I20260812 01:19:31.413965 5173 ts_tablet_manager.cc:1428] T d6a604a4264d4eef992e88f8c2cb2cae P 9b55292de1ce492b9e347c66f4cf0111: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:31.414155 5221 ts_tablet_manager.cc:1939] T 1401049ef7e2461e8491c205a2d505a5 P 0ffa5375dd2b431b852b9dd97c25d204: Deleting consensus metadata
I20260812 01:19:31.414572 4355 catalog_manager.cc:4841] TS 0ffa5375dd2b431b852b9dd97c25d204 (127.24.0.129:33711): tablet 1401049ef7e2461e8491c205a2d505a5 (table test-table [id=e8c44c2f0bcd4051930a3edd7154dfb7]) successfully deleted
I20260812 01:19:31.433315 5212 raft_consensus.cc:482] T d6a604a4264d4eef992e88f8c2cb2cae P 9b55292de1ce492b9e347c66f4cf0111 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 01:19:31.433398 5212 raft_consensus.cc:504] T d6a604a4264d4eef992e88f8c2cb2cae P 9b55292de1ce492b9e347c66f4cf0111 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ffa5375dd2b431b852b9dd97c25d204" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 33711 } } peers { permanent_uuid: "9b55292de1ce492b9e347c66f4cf0111" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39435 } } peers { permanent_uuid: "7107058860f74ceab1ce49475d781b6c" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 38125 } }
I20260812 01:19:31.433595 5212 leader_election.cc:288] T d6a604a4264d4eef992e88f8c2cb2cae P 9b55292de1ce492b9e347c66f4cf0111 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ffa5375dd2b431b852b9dd97c25d204 (127.24.0.129:33711), 7107058860f74ceab1ce49475d781b6c (127.24.0.131:38125)
I20260812 01:19:31.433808 5069 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6a604a4264d4eef992e88f8c2cb2cae" candidate_uuid: "9b55292de1ce492b9e347c66f4cf0111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7107058860f74ceab1ce49475d781b6c" is_pre_election: true
I20260812 01:19:31.433820 4636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6a604a4264d4eef992e88f8c2cb2cae" candidate_uuid: "9b55292de1ce492b9e347c66f4cf0111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ffa5375dd2b431b852b9dd97c25d204" is_pre_election: true
I20260812 01:19:31.433909 4636 raft_consensus.cc:2454] T d6a604a4264d4eef992e88f8c2cb2cae P 0ffa5375dd2b431b852b9dd97c25d204 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9b55292de1ce492b9e347c66f4cf0111 in term 0.
I20260812 01:19:31.433909 5069 raft_consensus.cc:2454] T d6a604a4264d4eef992e88f8c2cb2cae P 7107058860f74ceab1ce49475d781b6c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9b55292de1ce492b9e347c66f4cf0111 in term 0.
I20260812 01:19:31.434064 4748 leader_election.cc:302] T d6a604a4264d4eef992e88f8c2cb2cae P 9b55292de1ce492b9e347c66f4cf0111 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7107058860f74ceab1ce49475d781b6c, 9b55292de1ce492b9e347c66f4cf0111; no voters:
I20260812 01:19:31.434196 5212 raft_consensus.cc:2781] T d6a604a4264d4eef992e88f8c2cb2cae P 9b55292de1ce492b9e347c66f4cf0111 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 01:19:31.434233 5212 raft_consensus.cc:482] T d6a604a4264d4eef992e88f8c2cb2cae P 9b55292de1ce492b9e347c66f4cf0111 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 01:19:31.434265 5212 raft_consensus.cc:3037] T d6a604a4264d4eef992e88f8c2cb2cae P 9b55292de1ce492b9e347c66f4cf0111 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:19:31.434983 5212 raft_consensus.cc:504] T d6a604a4264d4eef992e88f8c2cb2cae P 9b55292de1ce492b9e347c66f4cf0111 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ffa5375dd2b431b852b9dd97c25d204" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 33711 } } peers { permanent_uuid: "9b55292de1ce492b9e347c66f4cf0111" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39435 } } peers { permanent_uuid: "7107058860f74ceab1ce49475d781b6c" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 38125 } }
I20260812 01:19:31.435154 5212 leader_election.cc:288] T d6a604a4264d4eef992e88f8c2cb2cae P 9b55292de1ce492b9e347c66f4cf0111 [CANDIDATE]: Term 1 election: Requested vote from peers 0ffa5375dd2b431b852b9dd97c25d204 (127.24.0.129:33711), 7107058860f74ceab1ce49475d781b6c (127.24.0.131:38125)
I20260812 01:19:31.435298 4636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6a604a4264d4eef992e88f8c2cb2cae" candidate_uuid: "9b55292de1ce492b9e347c66f4cf0111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ffa5375dd2b431b852b9dd97c25d204"
I20260812 01:19:31.435376 4636 raft_consensus.cc:3037] T d6a604a4264d4eef992e88f8c2cb2cae P 0ffa5375dd2b431b852b9dd97c25d204 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:19:31.435364 5069 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6a604a4264d4eef992e88f8c2cb2cae" candidate_uuid: "9b55292de1ce492b9e347c66f4cf0111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7107058860f74ceab1ce49475d781b6c"
I20260812 01:19:31.435431 5069 raft_consensus.cc:3037] T d6a604a4264d4eef992e88f8c2cb2cae P 7107058860f74ceab1ce49475d781b6c [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:19:31.436321 5069 raft_consensus.cc:2454] T d6a604a4264d4eef992e88f8c2cb2cae P 7107058860f74ceab1ce49475d781b6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b55292de1ce492b9e347c66f4cf0111 in term 1.
I20260812 01:19:31.436497 4748 leader_election.cc:302] T d6a604a4264d4eef992e88f8c2cb2cae P 9b55292de1ce492b9e347c66f4cf0111 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7107058860f74ceab1ce49475d781b6c, 9b55292de1ce492b9e347c66f4cf0111; no voters:
I20260812 01:19:31.436584 5212 raft_consensus.cc:2781] T d6a604a4264d4eef992e88f8c2cb2cae P 9b55292de1ce492b9e347c66f4cf0111 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:19:31.436640 5212 raft_consensus.cc:686] T d6a604a4264d4eef992e88f8c2cb2cae P 9b55292de1ce492b9e347c66f4cf0111 [term 1 LEADER]: Becoming Leader. State: Replica: 9b55292de1ce492b9e347c66f4cf0111, State: Running, Role: LEADER
I20260812 01:19:31.436738 4636 raft_consensus.cc:2454] T d6a604a4264d4eef992e88f8c2cb2cae P 0ffa5375dd2b431b852b9dd97c25d204 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b55292de1ce492b9e347c66f4cf0111 in term 1.
I20260812 01:19:31.436717 5212 consensus_queue.cc:237] T d6a604a4264d4eef992e88f8c2cb2cae P 9b55292de1ce492b9e347c66f4cf0111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ffa5375dd2b431b852b9dd97c25d204" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 33711 } } peers { permanent_uuid: "9b55292de1ce492b9e347c66f4cf0111" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39435 } } peers { permanent_uuid: "7107058860f74ceab1ce49475d781b6c" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 38125 } }
I20260812 01:19:31.437321 4372 catalog_manager.cc:5495] T d6a604a4264d4eef992e88f8c2cb2cae P 9b55292de1ce492b9e347c66f4cf0111 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9b55292de1ce492b9e347c66f4cf0111 (127.24.0.130). New cstate: current_term: 1 leader_uuid: "9b55292de1ce492b9e347c66f4cf0111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ffa5375dd2b431b852b9dd97c25d204" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 33711 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9b55292de1ce492b9e347c66f4cf0111" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39435 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7107058860f74ceab1ce49475d781b6c" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 38125 } health_report { overall_health: UNKNOWN } } }
Aug 12 01:19:31 dist-test-slave-t3q3 krb5kdc[4301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497570, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.0.190@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM
I20260812 01:19:31.464809 4462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:31.456908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44776 (local address 127.24.0.190:39395)
0812 01:19:31.456979 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:31.456984 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:31.457121 (+ 137us) server_negotiation.cc:407] Connection header received
0812 01:19:31.457188 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:31.457188 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:31.457198 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:31.457250 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:31.457878 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:31.458425 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:31.459455 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:31.459636 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:31.459688 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:31.462151 (+ 2463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:31.462153 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:31.462153 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:31.462159 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:31.463512 (+ 1353us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:31.463953 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:31.463954 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:31.463954 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:31.464002 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:31.464224 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:31.464225 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:31.464225 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:31.464325 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:31.464379 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:31.464593 (+ 214us) server_negotiation.cc:299] Negotiation successful
0812 01:19:31.464707 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":37,"server-negotiator.queue_time_us":47}
I20260812 01:19:31.464825 5239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:31.456820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:39395 (local address 127.0.0.1:44776)
0812 01:19:31.457037 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:31.457050 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:31.457126 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:31.457279 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:31.457280 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:31.457724 (+ 444us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:31.457791 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:31.457794 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:31.458510 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:31.458512 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:31.459368 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:31.459371 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:31.459716 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:31.459717 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:31.459779 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:31.460415 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:31.460421 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:31.462049 (+ 1628us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:19:31.463599 (+ 1550us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:31.463607 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:31.463609 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:31.463879 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:31.464086 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:31.464087 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:31.464087 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:31.464159 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:31.464395 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:19:31.464402 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:19:31.464456 (+ 54us) client_negotiation.cc:769] Sending connection context
0812 01:19:31.464531 (+ 75us) client_negotiation.cc:240] Negotiation successful
0812 01:19:31.464649 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":92,"threads_started":1}
I20260812 01:19:31.466964 4372 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.24.0.190@KRBTEST.COM'} at 127.0.0.1:44776:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260812 01:19:31.467191 4372 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 01:19:31.468796 4820 tablet_service.cc:1467] Processing CreateTablet for tablet 08fa5e76799943a5b6c1b89c392b4feb (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=13d3ba07d95246748428bd114c17b232]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 01:19:31.468840 5036 tablet_service.cc:1467] Processing CreateTablet for tablet 08fa5e76799943a5b6c1b89c392b4feb (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=13d3ba07d95246748428bd114c17b232]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 01:19:31.468981 4820 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08fa5e76799943a5b6c1b89c392b4feb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:19:31.469009 5036 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08fa5e76799943a5b6c1b89c392b4feb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:19:31.470150 5173 tablet_bootstrap.cc:492] T 08fa5e76799943a5b6c1b89c392b4feb P 9b55292de1ce492b9e347c66f4cf0111: Bootstrap starting.
I20260812 01:19:31.470436 5173 tablet_bootstrap.cc:654] T 08fa5e76799943a5b6c1b89c392b4feb P 9b55292de1ce492b9e347c66f4cf0111: Neither blocks nor log segments found. Creating new log.
I20260812 01:19:31.470615 5172 tablet_bootstrap.cc:492] T 08fa5e76799943a5b6c1b89c392b4feb P 7107058860f74ceab1ce49475d781b6c: Bootstrap starting.
I20260812 01:19:31.470750 4605 tablet_service.cc:1467] Processing CreateTablet for tablet 08fa5e76799943a5b6c1b89c392b4feb (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=13d3ba07d95246748428bd114c17b232]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 01:19:31.470902 4605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08fa5e76799943a5b6c1b89c392b4feb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:19:31.471060 5172 tablet_bootstrap.cc:654] T 08fa5e76799943a5b6c1b89c392b4feb P 7107058860f74ceab1ce49475d781b6c: Neither blocks nor log segments found. Creating new log.
I20260812 01:19:31.471261 5173 tablet_bootstrap.cc:492] T 08fa5e76799943a5b6c1b89c392b4feb P 9b55292de1ce492b9e347c66f4cf0111: No bootstrap required, opened a new log
I20260812 01:19:31.471305 5173 ts_tablet_manager.cc:1397] T 08fa5e76799943a5b6c1b89c392b4feb P 9b55292de1ce492b9e347c66f4cf0111: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 01:19:31.471444 5173 raft_consensus.cc:348] T 08fa5e76799943a5b6c1b89c392b4feb P 9b55292de1ce492b9e347c66f4cf0111 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ffa5375dd2b431b852b9dd97c25d204" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 33711 } } peers { permanent_uuid: "9b55292de1ce492b9e347c66f4cf0111" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39435 } } peers { permanent_uuid: "7107058860f74ceab1ce49475d781b6c" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 38125 } }
I20260812 01:19:31.471521 5173 raft_consensus.cc:374] T 08fa5e76799943a5b6c1b89c392b4feb P 9b55292de1ce492b9e347c66f4cf0111 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:19:31.471549 5173 raft_consensus.cc:729] T 08fa5e76799943a5b6c1b89c392b4feb P 9b55292de1ce492b9e347c66f4cf0111 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b55292de1ce492b9e347c66f4cf0111, State: Initialized, Role: FOLLOWER
I20260812 01:19:31.471602 5173 consensus_queue.cc:260] T 08fa5e76799943a5b6c1b89c392b4feb P 9b55292de1ce492b9e347c66f4cf0111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ffa5375dd2b431b852b9dd97c25d204" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 33711 } } peers { permanent_uuid: "9b55292de1ce492b9e347c66f4cf0111" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39435 } } peers { permanent_uuid: "7107058860f74ceab1ce49475d781b6c" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 38125 } }
I20260812 01:19:31.471696 5173 ts_tablet_manager.cc:1428] T 08fa5e76799943a5b6c1b89c392b4feb P 9b55292de1ce492b9e347c66f4cf0111: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:31.471717 5212 tablet_replica.cc:439] T 08fa5e76799943a5b6c1b89c392b4feb P 9b55292de1ce492b9e347c66f4cf0111: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ffa5375dd2b431b852b9dd97c25d204" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 33711 } } peers { permanent_uuid: "9b55292de1ce492b9e347c66f4cf0111" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39435 } } peers { permanent_uuid: "7107058860f74ceab1ce49475d781b6c" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 38125 } } }
I20260812 01:19:31.471784 5212 tablet_replica.cc:442] T 08fa5e76799943a5b6c1b89c392b4feb P 9b55292de1ce492b9e347c66f4cf0111: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 01:19:31.472500 5179 tablet_bootstrap.cc:492] T 08fa5e76799943a5b6c1b89c392b4feb P 0ffa5375dd2b431b852b9dd97c25d204: Bootstrap starting.
I20260812 01:19:31.472546 5172 tablet_bootstrap.cc:492] T 08fa5e76799943a5b6c1b89c392b4feb P 7107058860f74ceab1ce49475d781b6c: No bootstrap required, opened a new log
I20260812 01:19:31.472584 5172 ts_tablet_manager.cc:1397] T 08fa5e76799943a5b6c1b89c392b4feb P 7107058860f74ceab1ce49475d781b6c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 01:19:31.472743 5172 raft_consensus.cc:348] T 08fa5e76799943a5b6c1b89c392b4feb P 7107058860f74ceab1ce49475d781b6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ffa5375dd2b431b852b9dd97c25d204" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 33711 } } peers { permanent_uuid: "9b55292de1ce492b9e347c66f4cf0111" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39435 } } peers { permanent_uuid: "7107058860f74ceab1ce49475d781b6c" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 38125 } }
I20260812 01:19:31.472807 5172 raft_consensus.cc:374] T 08fa5e76799943a5b6c1b89c392b4feb P 7107058860f74ceab1ce49475d781b6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:19:31.472827 5172 raft_consensus.cc:729] T 08fa5e76799943a5b6c1b89c392b4feb P 7107058860f74ceab1ce49475d781b6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7107058860f74ceab1ce49475d781b6c, State: Initialized, Role: FOLLOWER
I20260812 01:19:31.472886 5172 consensus_queue.cc:260] T 08fa5e76799943a5b6c1b89c392b4feb P 7107058860f74ceab1ce49475d781b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ffa5375dd2b431b852b9dd97c25d204" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 33711 } } peers { permanent_uuid: "9b55292de1ce492b9e347c66f4cf0111" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39435 } } peers { permanent_uuid: "7107058860f74ceab1ce49475d781b6c" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 38125 } }
I20260812 01:19:31.472966 5179 tablet_bootstrap.cc:654] T 08fa5e76799943a5b6c1b89c392b4feb P 0ffa5375dd2b431b852b9dd97c25d204: Neither blocks nor log segments found. Creating new log.
I20260812 01:19:31.473014 5172 ts_tablet_manager.cc:1428] T 08fa5e76799943a5b6c1b89c392b4feb P 7107058860f74ceab1ce49475d781b6c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:31.473078 5183 tablet_replica.cc:439] T 08fa5e76799943a5b6c1b89c392b4feb P 7107058860f74ceab1ce49475d781b6c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ffa5375dd2b431b852b9dd97c25d204" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 33711 } } peers { permanent_uuid: "9b55292de1ce492b9e347c66f4cf0111" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39435 } } peers { permanent_uuid: "7107058860f74ceab1ce49475d781b6c" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 38125 } } }
I20260812 01:19:31.473192 5183 tablet_replica.cc:442] T 08fa5e76799943a5b6c1b89c392b4feb P 7107058860f74ceab1ce49475d781b6c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 01:19:31.473706 5179 tablet_bootstrap.cc:492] T 08fa5e76799943a5b6c1b89c392b4feb P 0ffa5375dd2b431b852b9dd97c25d204: No bootstrap required, opened a new log
I20260812 01:19:31.473760 5179 ts_tablet_manager.cc:1397] T 08fa5e76799943a5b6c1b89c392b4feb P 0ffa5375dd2b431b852b9dd97c25d204: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260812 01:19:31.473905 5179 raft_consensus.cc:348] T 08fa5e76799943a5b6c1b89c392b4feb P 0ffa5375dd2b431b852b9dd97c25d204 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ffa5375dd2b431b852b9dd97c25d204" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 33711 } } peers { permanent_uuid: "9b55292de1ce492b9e347c66f4cf0111" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39435 } } peers { permanent_uuid: "7107058860f74ceab1ce49475d781b6c" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 38125 } }
I20260812 01:19:31.473960 5179 raft_consensus.cc:374] T 08fa5e76799943a5b6c1b89c392b4feb P 0ffa5375dd2b431b852b9dd97c25d204 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:19:31.473974 5179 raft_consensus.cc:729] T 08fa5e76799943a5b6c1b89c392b4feb P 0ffa5375dd2b431b852b9dd97c25d204 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ffa5375dd2b431b852b9dd97c25d204, State: Initialized, Role: FOLLOWER
I20260812 01:19:31.474030 5179 consensus_queue.cc:260] T 08fa5e76799943a5b6c1b89c392b4feb P 0ffa5375dd2b431b852b9dd97c25d204 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ffa5375dd2b431b852b9dd97c25d204" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 33711 } } peers { permanent_uuid: "9b55292de1ce492b9e347c66f4cf0111" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39435 } } peers { permanent_uuid: "7107058860f74ceab1ce49475d781b6c" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 38125 } }
I20260812 01:19:31.474134 5179 ts_tablet_manager.cc:1428] T 08fa5e76799943a5b6c1b89c392b4feb P 0ffa5375dd2b431b852b9dd97c25d204: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:31.474211 5185 tablet_replica.cc:439] T 08fa5e76799943a5b6c1b89c392b4feb P 0ffa5375dd2b431b852b9dd97c25d204: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ffa5375dd2b431b852b9dd97c25d204" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 33711 } } peers { permanent_uuid: "9b55292de1ce492b9e347c66f4cf0111" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39435 } } peers { permanent_uuid: "7107058860f74ceab1ce49475d781b6c" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 38125 } } }
I20260812 01:19:31.474282 5185 tablet_replica.cc:442] T 08fa5e76799943a5b6c1b89c392b4feb P 0ffa5375dd2b431b852b9dd97c25d204: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 01:19:31.620281 5212 raft_consensus.cc:482] T 08fa5e76799943a5b6c1b89c392b4feb P 9b55292de1ce492b9e347c66f4cf0111 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 01:19:31.620390 5212 raft_consensus.cc:504] T 08fa5e76799943a5b6c1b89c392b4feb P 9b55292de1ce492b9e347c66f4cf0111 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ffa5375dd2b431b852b9dd97c25d204" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 33711 } } peers { permanent_uuid: "9b55292de1ce492b9e347c66f4cf0111" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39435 } } peers { permanent_uuid: "7107058860f74ceab1ce49475d781b6c" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 38125 } }
I20260812 01:19:31.620549 5212 leader_election.cc:288] T 08fa5e76799943a5b6c1b89c392b4feb P 9b55292de1ce492b9e347c66f4cf0111 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ffa5375dd2b431b852b9dd97c25d204 (127.24.0.129:33711), 7107058860f74ceab1ce49475d781b6c (127.24.0.131:38125)
I20260812 01:19:31.620800 4636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08fa5e76799943a5b6c1b89c392b4feb" candidate_uuid: "9b55292de1ce492b9e347c66f4cf0111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ffa5375dd2b431b852b9dd97c25d204" is_pre_election: true
I20260812 01:19:31.620800 5069 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08fa5e76799943a5b6c1b89c392b4feb" candidate_uuid: "9b55292de1ce492b9e347c66f4cf0111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7107058860f74ceab1ce49475d781b6c" is_pre_election: true
I20260812 01:19:31.620913 5069 raft_consensus.cc:2454] T 08fa5e76799943a5b6c1b89c392b4feb P 7107058860f74ceab1ce49475d781b6c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9b55292de1ce492b9e347c66f4cf0111 in term 0.
I20260812 01:19:31.620913 4636 raft_consensus.cc:2454] T 08fa5e76799943a5b6c1b89c392b4feb P 0ffa5375dd2b431b852b9dd97c25d204 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9b55292de1ce492b9e347c66f4cf0111 in term 0.
I20260812 01:19:31.621111 4748 leader_election.cc:302] T 08fa5e76799943a5b6c1b89c392b4feb P 9b55292de1ce492b9e347c66f4cf0111 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7107058860f74ceab1ce49475d781b6c, 9b55292de1ce492b9e347c66f4cf0111; no voters:
I20260812 01:19:31.621235 5212 raft_consensus.cc:2781] T 08fa5e76799943a5b6c1b89c392b4feb P 9b55292de1ce492b9e347c66f4cf0111 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 01:19:31.621320 5212 raft_consensus.cc:482] T 08fa5e76799943a5b6c1b89c392b4feb P 9b55292de1ce492b9e347c66f4cf0111 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 01:19:31.621352 5212 raft_consensus.cc:3037] T 08fa5e76799943a5b6c1b89c392b4feb P 9b55292de1ce492b9e347c66f4cf0111 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:19:31.622048 5212 raft_consensus.cc:504] T 08fa5e76799943a5b6c1b89c392b4feb P 9b55292de1ce492b9e347c66f4cf0111 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ffa5375dd2b431b852b9dd97c25d204" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 33711 } } peers { permanent_uuid: "9b55292de1ce492b9e347c66f4cf0111" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39435 } } peers { permanent_uuid: "7107058860f74ceab1ce49475d781b6c" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 38125 } }
I20260812 01:19:31.622236 5212 leader_election.cc:288] T 08fa5e76799943a5b6c1b89c392b4feb P 9b55292de1ce492b9e347c66f4cf0111 [CANDIDATE]: Term 1 election: Requested vote from peers 0ffa5375dd2b431b852b9dd97c25d204 (127.24.0.129:33711), 7107058860f74ceab1ce49475d781b6c (127.24.0.131:38125)
I20260812 01:19:31.622386 5069 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08fa5e76799943a5b6c1b89c392b4feb" candidate_uuid: "9b55292de1ce492b9e347c66f4cf0111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7107058860f74ceab1ce49475d781b6c"
I20260812 01:19:31.622466 5069 raft_consensus.cc:3037] T 08fa5e76799943a5b6c1b89c392b4feb P 7107058860f74ceab1ce49475d781b6c [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:19:31.622539 4636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08fa5e76799943a5b6c1b89c392b4feb" candidate_uuid: "9b55292de1ce492b9e347c66f4cf0111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ffa5375dd2b431b852b9dd97c25d204"
I20260812 01:19:31.622643 4636 raft_consensus.cc:3037] T 08fa5e76799943a5b6c1b89c392b4feb P 0ffa5375dd2b431b852b9dd97c25d204 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:19:31.623138 5069 raft_consensus.cc:2454] T 08fa5e76799943a5b6c1b89c392b4feb P 7107058860f74ceab1ce49475d781b6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b55292de1ce492b9e347c66f4cf0111 in term 1.
I20260812 01:19:31.623304 4748 leader_election.cc:302] T 08fa5e76799943a5b6c1b89c392b4feb P 9b55292de1ce492b9e347c66f4cf0111 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7107058860f74ceab1ce49475d781b6c, 9b55292de1ce492b9e347c66f4cf0111; no voters:
I20260812 01:19:31.623409 5212 raft_consensus.cc:2781] T 08fa5e76799943a5b6c1b89c392b4feb P 9b55292de1ce492b9e347c66f4cf0111 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:19:31.623415 4636 raft_consensus.cc:2454] T 08fa5e76799943a5b6c1b89c392b4feb P 0ffa5375dd2b431b852b9dd97c25d204 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b55292de1ce492b9e347c66f4cf0111 in term 1.
I20260812 01:19:31.623482 5212 raft_consensus.cc:686] T 08fa5e76799943a5b6c1b89c392b4feb P 9b55292de1ce492b9e347c66f4cf0111 [term 1 LEADER]: Becoming Leader. State: Replica: 9b55292de1ce492b9e347c66f4cf0111, State: Running, Role: LEADER
I20260812 01:19:31.623538 5212 consensus_queue.cc:237] T 08fa5e76799943a5b6c1b89c392b4feb P 9b55292de1ce492b9e347c66f4cf0111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ffa5375dd2b431b852b9dd97c25d204" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 33711 } } peers { permanent_uuid: "9b55292de1ce492b9e347c66f4cf0111" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39435 } } peers { permanent_uuid: "7107058860f74ceab1ce49475d781b6c" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 38125 } }
I20260812 01:19:31.623842 5189 tablet_replica.cc:439] T 08fa5e76799943a5b6c1b89c392b4feb P 9b55292de1ce492b9e347c66f4cf0111: TxnStatusTablet state changed. Reason: New leader 9b55292de1ce492b9e347c66f4cf0111. Latest consensus state: current_term: 1 leader_uuid: "9b55292de1ce492b9e347c66f4cf0111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ffa5375dd2b431b852b9dd97c25d204" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 33711 } } peers { permanent_uuid: "9b55292de1ce492b9e347c66f4cf0111" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39435 } } peers { permanent_uuid: "7107058860f74ceab1ce49475d781b6c" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 38125 } } }
I20260812 01:19:31.623936 5189 tablet_replica.cc:442] T 08fa5e76799943a5b6c1b89c392b4feb P 9b55292de1ce492b9e347c66f4cf0111: This TxnStatusTablet replica's current role is: LEADER
I20260812 01:19:31.624154 5247 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260812 01:19:31.624290 5247 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260812 01:19:31.624235 4375 catalog_manager.cc:5495] T 08fa5e76799943a5b6c1b89c392b4feb P 9b55292de1ce492b9e347c66f4cf0111 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9b55292de1ce492b9e347c66f4cf0111 (127.24.0.130). New cstate: current_term: 1 leader_uuid: "9b55292de1ce492b9e347c66f4cf0111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ffa5375dd2b431b852b9dd97c25d204" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 33711 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9b55292de1ce492b9e347c66f4cf0111" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39435 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7107058860f74ceab1ce49475d781b6c" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 38125 } health_report { overall_health: UNKNOWN } } }
I20260812 01:19:31.655401 5170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:31.653047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40254 (local address 127.24.0.130:39435)
0812 01:19:31.653110 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:31.653113 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:31.653169 (+ 56us) server_negotiation.cc:407] Connection header received
0812 01:19:31.653259 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:31.653260 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:31.653272 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:31.653343 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:19:31.653486 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:31.653951 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:31.654716 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:31.654900 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:31.654975 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:31.655150 (+ 175us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:19:31.655186 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:19:31.655241 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:31.655311 (+ 70us) server_negotiation.cc:299] Negotiation successful
0812 01:19:31.655330 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 01:19:31.655424 5239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:31.652970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.130:39435 (local address 127.0.0.1:40254)
0812 01:19:31.653116 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:31.653128 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:31.653194 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:31.653352 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:31.653353 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:31.653354 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 01:19:31.653406 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:31.653408 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:31.654035 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:31.654036 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:31.654625 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:31.654627 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:31.654975 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:31.654976 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:31.655034 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:31.655060 (+ 26us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 01:19:31.655255 (+ 195us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 01:19:31.655265 (+ 10us) client_negotiation.cc:769] Sending connection context
0812 01:19:31.655309 (+ 44us) client_negotiation.cc:240] Negotiation successful
0812 01:19:31.655355 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104}
I20260812 01:19:31.656418 4636 raft_consensus.cc:1261] T 08fa5e76799943a5b6c1b89c392b4feb P 0ffa5375dd2b431b852b9dd97c25d204 [term 1 FOLLOWER]: Refusing update from remote peer 9b55292de1ce492b9e347c66f4cf0111: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:19:31.656437 5069 raft_consensus.cc:1261] T 08fa5e76799943a5b6c1b89c392b4feb P 7107058860f74ceab1ce49475d781b6c [term 1 FOLLOWER]: Refusing update from remote peer 9b55292de1ce492b9e347c66f4cf0111: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:19:31.656663 5212 consensus_queue.cc:1035] T 08fa5e76799943a5b6c1b89c392b4feb P 9b55292de1ce492b9e347c66f4cf0111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ffa5375dd2b431b852b9dd97c25d204" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 33711 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:19:31.656764 5189 consensus_queue.cc:1035] T 08fa5e76799943a5b6c1b89c392b4feb P 9b55292de1ce492b9e347c66f4cf0111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7107058860f74ceab1ce49475d781b6c" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 38125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:19:31.657337 5185 tablet_replica.cc:439] T 08fa5e76799943a5b6c1b89c392b4feb P 0ffa5375dd2b431b852b9dd97c25d204: TxnStatusTablet state changed. Reason: New leader 9b55292de1ce492b9e347c66f4cf0111. Latest consensus state: current_term: 1 leader_uuid: "9b55292de1ce492b9e347c66f4cf0111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ffa5375dd2b431b852b9dd97c25d204" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 33711 } } peers { permanent_uuid: "9b55292de1ce492b9e347c66f4cf0111" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39435 } } peers { permanent_uuid: "7107058860f74ceab1ce49475d781b6c" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 38125 } } }
I20260812 01:19:31.657433 5185 tablet_replica.cc:442] T 08fa5e76799943a5b6c1b89c392b4feb P 0ffa5375dd2b431b852b9dd97c25d204: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 01:19:31.657560 5183 tablet_replica.cc:439] T 08fa5e76799943a5b6c1b89c392b4feb P 7107058860f74ceab1ce49475d781b6c: TxnStatusTablet state changed. Reason: New leader 9b55292de1ce492b9e347c66f4cf0111. Latest consensus state: current_term: 1 leader_uuid: "9b55292de1ce492b9e347c66f4cf0111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ffa5375dd2b431b852b9dd97c25d204" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 33711 } } peers { permanent_uuid: "9b55292de1ce492b9e347c66f4cf0111" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39435 } } peers { permanent_uuid: "7107058860f74ceab1ce49475d781b6c" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 38125 } } }
I20260812 01:19:31.657666 5183 tablet_replica.cc:442] T 08fa5e76799943a5b6c1b89c392b4feb P 7107058860f74ceab1ce49475d781b6c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 01:19:31.658066 5182 tablet_replica.cc:439] T 08fa5e76799943a5b6c1b89c392b4feb P 9b55292de1ce492b9e347c66f4cf0111: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9b55292de1ce492b9e347c66f4cf0111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ffa5375dd2b431b852b9dd97c25d204" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 33711 } } peers { permanent_uuid: "9b55292de1ce492b9e347c66f4cf0111" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39435 } } peers { permanent_uuid: "7107058860f74ceab1ce49475d781b6c" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 38125 } } }
I20260812 01:19:31.658165 5182 tablet_replica.cc:442] T 08fa5e76799943a5b6c1b89c392b4feb P 9b55292de1ce492b9e347c66f4cf0111: This TxnStatusTablet replica's current role is: LEADER
I20260812 01:19:31.658293 5185 tablet_replica.cc:439] T 08fa5e76799943a5b6c1b89c392b4feb P 0ffa5375dd2b431b852b9dd97c25d204: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9b55292de1ce492b9e347c66f4cf0111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ffa5375dd2b431b852b9dd97c25d204" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 33711 } } peers { permanent_uuid: "9b55292de1ce492b9e347c66f4cf0111" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39435 } } peers { permanent_uuid: "7107058860f74ceab1ce49475d781b6c" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 38125 } } }
I20260812 01:19:31.658380 5185 tablet_replica.cc:442] T 08fa5e76799943a5b6c1b89c392b4feb P 0ffa5375dd2b431b852b9dd97c25d204: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 01:19:31.658823 5182 tablet_replica.cc:439] T 08fa5e76799943a5b6c1b89c392b4feb P 9b55292de1ce492b9e347c66f4cf0111: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9b55292de1ce492b9e347c66f4cf0111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ffa5375dd2b431b852b9dd97c25d204" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 33711 } } peers { permanent_uuid: "9b55292de1ce492b9e347c66f4cf0111" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39435 } } peers { permanent_uuid: "7107058860f74ceab1ce49475d781b6c" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 38125 } } }
I20260812 01:19:31.658903 5182 tablet_replica.cc:442] T 08fa5e76799943a5b6c1b89c392b4feb P 9b55292de1ce492b9e347c66f4cf0111: This TxnStatusTablet replica's current role is: LEADER
I20260812 01:19:31.659277 5199 mvcc.cc:205] Tried to move back new op lower bound from 7317494053502885888 to 7317494053370257408. Current Snapshot: MvccSnapshot[applied={T|T < 7317494053502885888}]
I20260812 01:19:31.659387 5183 tablet_replica.cc:439] T 08fa5e76799943a5b6c1b89c392b4feb P 7107058860f74ceab1ce49475d781b6c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9b55292de1ce492b9e347c66f4cf0111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ffa5375dd2b431b852b9dd97c25d204" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 33711 } } peers { permanent_uuid: "9b55292de1ce492b9e347c66f4cf0111" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39435 } } peers { permanent_uuid: "7107058860f74ceab1ce49475d781b6c" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 38125 } } }
I20260812 01:19:31.659459 5183 tablet_replica.cc:442] T 08fa5e76799943a5b6c1b89c392b4feb P 7107058860f74ceab1ce49475d781b6c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 01:19:31.667670 5170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:31.665354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.130:41667 (local address 127.24.0.130:39435)
0812 01:19:31.665423 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:31.665426 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:31.665558 (+ 132us) server_negotiation.cc:407] Connection header received
0812 01:19:31.665628 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:31.665629 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:31.665637 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:31.665682 (+ 45us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:19:31.665781 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:31.666221 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:31.666953 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:31.667154 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:31.667238 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:31.667390 (+ 152us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:19:31.667431 (+ 41us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:19:31.667493 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:31.667555 (+ 62us) server_negotiation.cc:299] Negotiation successful
0812 01:19:31.667591 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":15,"server-negotiator.queue_time_us":45}
I20260812 01:19:31.667692 5258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:31.665283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.130:39435 (local address 127.24.0.130:41667)
0812 01:19:31.665491 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:31.665500 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:31.665571 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:31.665684 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:31.665685 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:31.665686 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 01:19:31.665726 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:31.665727 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:31.666302 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:31.666303 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:31.666854 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:31.666855 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:31.667239 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:31.667240 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:31.667290 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:31.667304 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 01:19:31.667495 (+ 191us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 01:19:31.667497 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 01:19:31.667550 (+ 53us) client_negotiation.cc:240] Negotiation successful
0812 01:19:31.667610 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"mutex_wait_us":3,"thread_start_us":109,"threads_started":1}
I20260812 01:19:31.669978 5069 raft_consensus.cc:1261] T d6a604a4264d4eef992e88f8c2cb2cae P 7107058860f74ceab1ce49475d781b6c [term 1 FOLLOWER]: Refusing update from remote peer 9b55292de1ce492b9e347c66f4cf0111: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:19:31.670150 4636 raft_consensus.cc:1261] T d6a604a4264d4eef992e88f8c2cb2cae P 0ffa5375dd2b431b852b9dd97c25d204 [term 1 FOLLOWER]: Refusing update from remote peer 9b55292de1ce492b9e347c66f4cf0111: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:19:31.670207 5212 consensus_queue.cc:1035] T d6a604a4264d4eef992e88f8c2cb2cae P 9b55292de1ce492b9e347c66f4cf0111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7107058860f74ceab1ce49475d781b6c" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 38125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:19:31.670332 5212 consensus_queue.cc:1035] T d6a604a4264d4eef992e88f8c2cb2cae P 9b55292de1ce492b9e347c66f4cf0111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ffa5375dd2b431b852b9dd97c25d204" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 33711 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:19:31.672603 5198 mvcc.cc:205] Tried to move back new op lower bound from 7317494053558792192 to 7317494052605272064. Current Snapshot: MvccSnapshot[applied={T|T < 7317494053558792192}]
I20260812 01:19:31.684589 4372 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44772:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:19:31.684702 4372 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44772:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:19:31.685353 4372 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8cf578e52d544713ab6c35cb1333039f: Sending DeleteTablet for 3 replicas of tablet d6a604a4264d4eef992e88f8c2cb2cae
I20260812 01:19:31.685621 5036 tablet_service.cc:1514] Processing DeleteTablet for tablet d6a604a4264d4eef992e88f8c2cb2cae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:19:31 UTC) from {username='oryx', principal='oryx/127.24.0.190@KRBTEST.COM'} at 127.0.0.1:34842
I20260812 01:19:31.685743 5218 tablet_replica.cc:330] T d6a604a4264d4eef992e88f8c2cb2cae P 7107058860f74ceab1ce49475d781b6c: stopping tablet replica
I20260812 01:19:31.685719 4605 tablet_service.cc:1514] Processing DeleteTablet for tablet d6a604a4264d4eef992e88f8c2cb2cae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:19:31 UTC) from {username='oryx', principal='oryx/127.24.0.190@KRBTEST.COM'} at 127.0.0.1:55444
I20260812 01:19:31.685793 5218 raft_consensus.cc:2229] T d6a604a4264d4eef992e88f8c2cb2cae P 7107058860f74ceab1ce49475d781b6c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 01:19:31.685804 5221 tablet_replica.cc:330] T d6a604a4264d4eef992e88f8c2cb2cae P 0ffa5375dd2b431b852b9dd97c25d204: stopping tablet replica
I20260812 01:19:31.685837 5218 raft_consensus.cc:2258] T d6a604a4264d4eef992e88f8c2cb2cae P 7107058860f74ceab1ce49475d781b6c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:19:31.685847 5221 raft_consensus.cc:2229] T d6a604a4264d4eef992e88f8c2cb2cae P 0ffa5375dd2b431b852b9dd97c25d204 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 01:19:31.685882 5221 raft_consensus.cc:2258] T d6a604a4264d4eef992e88f8c2cb2cae P 0ffa5375dd2b431b852b9dd97c25d204 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:19:31.686031 4819 tablet_service.cc:1514] Processing DeleteTablet for tablet d6a604a4264d4eef992e88f8c2cb2cae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:19:31 UTC) from {username='oryx', principal='oryx/127.24.0.190@KRBTEST.COM'} at 127.0.0.1:40224
I20260812 01:19:31.686126 5220 tablet_replica.cc:330] T d6a604a4264d4eef992e88f8c2cb2cae P 9b55292de1ce492b9e347c66f4cf0111: stopping tablet replica
I20260812 01:19:31.686178 5220 raft_consensus.cc:2229] T d6a604a4264d4eef992e88f8c2cb2cae P 9b55292de1ce492b9e347c66f4cf0111 [term 1 LEADER]: Raft consensus shutting down.
I20260812 01:19:31.686231 5220 raft_consensus.cc:2258] T d6a604a4264d4eef992e88f8c2cb2cae P 9b55292de1ce492b9e347c66f4cf0111 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:19:31.686247 5221 ts_tablet_manager.cc:1905] T d6a604a4264d4eef992e88f8c2cb2cae P 0ffa5375dd2b431b852b9dd97c25d204: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:19:31.686342 5218 ts_tablet_manager.cc:1905] T d6a604a4264d4eef992e88f8c2cb2cae P 7107058860f74ceab1ce49475d781b6c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:19:31.687758 5218 ts_tablet_manager.cc:1918] T d6a604a4264d4eef992e88f8c2cb2cae P 7107058860f74ceab1ce49475d781b6c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 01:19:31.687808 5218 log.cc:1196] T d6a604a4264d4eef992e88f8c2cb2cae P 7107058860f74ceab1ce49475d781b6c: Deleting WAL directory at /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/minicluster-data/ts-2/wal/wals/d6a604a4264d4eef992e88f8c2cb2cae
I20260812 01:19:31.687979 5218 ts_tablet_manager.cc:1939] T d6a604a4264d4eef992e88f8c2cb2cae P 7107058860f74ceab1ce49475d781b6c: Deleting consensus metadata
I20260812 01:19:31.688316 5220 ts_tablet_manager.cc:1905] T d6a604a4264d4eef992e88f8c2cb2cae P 9b55292de1ce492b9e347c66f4cf0111: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:19:31.688329 4356 catalog_manager.cc:4841] TS 7107058860f74ceab1ce49475d781b6c (127.24.0.131:38125): tablet d6a604a4264d4eef992e88f8c2cb2cae (table test-table [id=12faae1386c84bc9af3c5e1c32389747]) successfully deleted
I20260812 01:19:31.689926 5220 ts_tablet_manager.cc:1918] T d6a604a4264d4eef992e88f8c2cb2cae P 9b55292de1ce492b9e347c66f4cf0111: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 01:19:31.689975 5220 log.cc:1196] T d6a604a4264d4eef992e88f8c2cb2cae P 9b55292de1ce492b9e347c66f4cf0111: Deleting WAL directory at /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/minicluster-data/ts-1/wal/wals/d6a604a4264d4eef992e88f8c2cb2cae
I20260812 01:19:31.690177 5220 ts_tablet_manager.cc:1939] T d6a604a4264d4eef992e88f8c2cb2cae P 9b55292de1ce492b9e347c66f4cf0111: Deleting consensus metadata
I20260812 01:19:31.690485 4355 catalog_manager.cc:4841] TS 9b55292de1ce492b9e347c66f4cf0111 (127.24.0.130:39435): tablet d6a604a4264d4eef992e88f8c2cb2cae (table test-table [id=12faae1386c84bc9af3c5e1c32389747]) successfully deleted
I20260812 01:19:31.690712 5221 ts_tablet_manager.cc:1918] T d6a604a4264d4eef992e88f8c2cb2cae P 0ffa5375dd2b431b852b9dd97c25d204: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 01:19:31.690785 5221 log.cc:1196] T d6a604a4264d4eef992e88f8c2cb2cae P 0ffa5375dd2b431b852b9dd97c25d204: Deleting WAL directory at /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/minicluster-data/ts-0/wal/wals/d6a604a4264d4eef992e88f8c2cb2cae
I20260812 01:19:31.691007 5221 ts_tablet_manager.cc:1939] T d6a604a4264d4eef992e88f8c2cb2cae P 0ffa5375dd2b431b852b9dd97c25d204: Deleting consensus metadata
I20260812 01:19:31.691422 4355 catalog_manager.cc:4841] TS 0ffa5375dd2b431b852b9dd97c25d204 (127.24.0.129:33711): tablet d6a604a4264d4eef992e88f8c2cb2cae (table test-table [id=12faae1386c84bc9af3c5e1c32389747]) successfully deleted
Aug 12 01:19:31 dist-test-slave-t3q3 krb5kdc[4301](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497571, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 01:19:31 dist-test-slave-t3q3 krb5kdc[4301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:19:31 dist-test-slave-t3q3 krb5kdc[4301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
W20260812 01:19:31.710696 4462 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:19:31.704745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44782 (local address 127.24.0.190:39395)
0812 01:19:31.704814 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:31.704817 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:31.704920 (+ 103us) server_negotiation.cc:407] Connection header received
0812 01:19:31.704959 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:31.704961 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:31.704973 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:31.705049 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:31.705627 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:31.706391 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:31.707180 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:31.707470 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:31.707547 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:31.710626 (+ 3079us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44782: BlockingRecv error: recv got EOF from 127.0.0.1:44782 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 12 01:19:31 dist-test-slave-t3q3 krb5kdc[4301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497571, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM
I20260812 01:19:31.719722 4462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:31.711764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44788 (local address 127.24.0.190:39395)
0812 01:19:31.711805 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:31.711807 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:31.711898 (+ 91us) server_negotiation.cc:407] Connection header received
0812 01:19:31.711955 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:31.711955 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:31.711964 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:31.712013 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:31.712693 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:31.713224 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:31.714237 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:31.714427 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:31.714592 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:31.717085 (+ 2493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:31.717086 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:31.717087 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:31.717096 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:31.718548 (+ 1452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:31.718920 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:31.718921 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:31.718921 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:31.718980 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:31.719186 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:31.719187 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:31.719187 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:31.719303 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:31.719405 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:31.719531 (+ 126us) server_negotiation.cc:299] Negotiation successful
0812 01:19:31.719609 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 01:19:31.721732 4372 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44788:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 01:19:31.721917 4372 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 01:19:31.723522 4605 tablet_service.cc:1467] Processing CreateTablet for tablet d80c3945b82b43a6bfec8762aa7ef279 (DEFAULT_TABLE table=test-table [id=d943e1d35c574850bdbb2011a3c5576b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:19:31.723527 5036 tablet_service.cc:1467] Processing CreateTablet for tablet d80c3945b82b43a6bfec8762aa7ef279 (DEFAULT_TABLE table=test-table [id=d943e1d35c574850bdbb2011a3c5576b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:19:31.723582 4819 tablet_service.cc:1467] Processing CreateTablet for tablet d80c3945b82b43a6bfec8762aa7ef279 (DEFAULT_TABLE table=test-table [id=d943e1d35c574850bdbb2011a3c5576b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:19:31.723670 5036 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d80c3945b82b43a6bfec8762aa7ef279. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:19:31.723670 4605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d80c3945b82b43a6bfec8762aa7ef279. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:19:31.723726 4819 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d80c3945b82b43a6bfec8762aa7ef279. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:19:31.725047 5172 tablet_bootstrap.cc:492] T d80c3945b82b43a6bfec8762aa7ef279 P 7107058860f74ceab1ce49475d781b6c: Bootstrap starting.
I20260812 01:19:31.725057 5179 tablet_bootstrap.cc:492] T d80c3945b82b43a6bfec8762aa7ef279 P 0ffa5375dd2b431b852b9dd97c25d204: Bootstrap starting.
I20260812 01:19:31.725219 5173 tablet_bootstrap.cc:492] T d80c3945b82b43a6bfec8762aa7ef279 P 9b55292de1ce492b9e347c66f4cf0111: Bootstrap starting.
I20260812 01:19:31.725490 5179 tablet_bootstrap.cc:654] T d80c3945b82b43a6bfec8762aa7ef279 P 0ffa5375dd2b431b852b9dd97c25d204: Neither blocks nor log segments found. Creating new log.
I20260812 01:19:31.725492 5172 tablet_bootstrap.cc:654] T d80c3945b82b43a6bfec8762aa7ef279 P 7107058860f74ceab1ce49475d781b6c: Neither blocks nor log segments found. Creating new log.
I20260812 01:19:31.725526 5173 tablet_bootstrap.cc:654] T d80c3945b82b43a6bfec8762aa7ef279 P 9b55292de1ce492b9e347c66f4cf0111: Neither blocks nor log segments found. Creating new log.
I20260812 01:19:31.726362 5173 tablet_bootstrap.cc:492] T d80c3945b82b43a6bfec8762aa7ef279 P 9b55292de1ce492b9e347c66f4cf0111: No bootstrap required, opened a new log
I20260812 01:19:31.726419 5173 ts_tablet_manager.cc:1397] T d80c3945b82b43a6bfec8762aa7ef279 P 9b55292de1ce492b9e347c66f4cf0111: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 01:19:31.726470 5172 tablet_bootstrap.cc:492] T d80c3945b82b43a6bfec8762aa7ef279 P 7107058860f74ceab1ce49475d781b6c: No bootstrap required, opened a new log
I20260812 01:19:31.726531 5172 ts_tablet_manager.cc:1397] T d80c3945b82b43a6bfec8762aa7ef279 P 7107058860f74ceab1ce49475d781b6c: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 01:19:31.726549 5173 raft_consensus.cc:348] T d80c3945b82b43a6bfec8762aa7ef279 P 9b55292de1ce492b9e347c66f4cf0111 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ffa5375dd2b431b852b9dd97c25d204" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 33711 } } peers { permanent_uuid: "9b55292de1ce492b9e347c66f4cf0111" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39435 } } peers { permanent_uuid: "7107058860f74ceab1ce49475d781b6c" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 38125 } }
I20260812 01:19:31.726635 5173 raft_consensus.cc:374] T d80c3945b82b43a6bfec8762aa7ef279 P 9b55292de1ce492b9e347c66f4cf0111 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:19:31.726665 5173 raft_consensus.cc:729] T d80c3945b82b43a6bfec8762aa7ef279 P 9b55292de1ce492b9e347c66f4cf0111 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b55292de1ce492b9e347c66f4cf0111, State: Initialized, Role: FOLLOWER
I20260812 01:19:31.726709 5173 consensus_queue.cc:260] T d80c3945b82b43a6bfec8762aa7ef279 P 9b55292de1ce492b9e347c66f4cf0111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ffa5375dd2b431b852b9dd97c25d204" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 33711 } } peers { permanent_uuid: "9b55292de1ce492b9e347c66f4cf0111" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39435 } } peers { permanent_uuid: "7107058860f74ceab1ce49475d781b6c" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 38125 } }
I20260812 01:19:31.726693 5172 raft_consensus.cc:348] T d80c3945b82b43a6bfec8762aa7ef279 P 7107058860f74ceab1ce49475d781b6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ffa5375dd2b431b852b9dd97c25d204" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 33711 } } peers { permanent_uuid: "9b55292de1ce492b9e347c66f4cf0111" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39435 } } peers { permanent_uuid: "7107058860f74ceab1ce49475d781b6c" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 38125 } }
I20260812 01:19:31.726760 5172 raft_consensus.cc:374] T d80c3945b82b43a6bfec8762aa7ef279 P 7107058860f74ceab1ce49475d781b6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:19:31.726779 5172 raft_consensus.cc:729] T d80c3945b82b43a6bfec8762aa7ef279 P 7107058860f74ceab1ce49475d781b6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7107058860f74ceab1ce49475d781b6c, State: Initialized, Role: FOLLOWER
I20260812 01:19:31.726804 5173 ts_tablet_manager.cc:1428] T d80c3945b82b43a6bfec8762aa7ef279 P 9b55292de1ce492b9e347c66f4cf0111: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 01:19:31.726823 5172 consensus_queue.cc:260] T d80c3945b82b43a6bfec8762aa7ef279 P 7107058860f74ceab1ce49475d781b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ffa5375dd2b431b852b9dd97c25d204" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 33711 } } peers { permanent_uuid: "9b55292de1ce492b9e347c66f4cf0111" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39435 } } peers { permanent_uuid: "7107058860f74ceab1ce49475d781b6c" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 38125 } }
I20260812 01:19:31.726917 5172 ts_tablet_manager.cc:1428] T d80c3945b82b43a6bfec8762aa7ef279 P 7107058860f74ceab1ce49475d781b6c: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 01:19:31.728636 5179 tablet_bootstrap.cc:492] T d80c3945b82b43a6bfec8762aa7ef279 P 0ffa5375dd2b431b852b9dd97c25d204: No bootstrap required, opened a new log
I20260812 01:19:31.728708 5179 ts_tablet_manager.cc:1397] T d80c3945b82b43a6bfec8762aa7ef279 P 0ffa5375dd2b431b852b9dd97c25d204: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260812 01:19:31.728876 5179 raft_consensus.cc:348] T d80c3945b82b43a6bfec8762aa7ef279 P 0ffa5375dd2b431b852b9dd97c25d204 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ffa5375dd2b431b852b9dd97c25d204" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 33711 } } peers { permanent_uuid: "9b55292de1ce492b9e347c66f4cf0111" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39435 } } peers { permanent_uuid: "7107058860f74ceab1ce49475d781b6c" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 38125 } }
I20260812 01:19:31.728967 5179 raft_consensus.cc:374] T d80c3945b82b43a6bfec8762aa7ef279 P 0ffa5375dd2b431b852b9dd97c25d204 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:19:31.728988 5179 raft_consensus.cc:729] T d80c3945b82b43a6bfec8762aa7ef279 P 0ffa5375dd2b431b852b9dd97c25d204 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ffa5375dd2b431b852b9dd97c25d204, State: Initialized, Role: FOLLOWER
I20260812 01:19:31.729041 5179 consensus_queue.cc:260] T d80c3945b82b43a6bfec8762aa7ef279 P 0ffa5375dd2b431b852b9dd97c25d204 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ffa5375dd2b431b852b9dd97c25d204" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 33711 } } peers { permanent_uuid: "9b55292de1ce492b9e347c66f4cf0111" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39435 } } peers { permanent_uuid: "7107058860f74ceab1ce49475d781b6c" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 38125 } }
I20260812 01:19:31.729156 5179 ts_tablet_manager.cc:1428] T d80c3945b82b43a6bfec8762aa7ef279 P 0ffa5375dd2b431b852b9dd97c25d204: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260812 01:19:31.791198 5212 raft_consensus.cc:482] T d80c3945b82b43a6bfec8762aa7ef279 P 9b55292de1ce492b9e347c66f4cf0111 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 01:19:31.791293 5212 raft_consensus.cc:504] T d80c3945b82b43a6bfec8762aa7ef279 P 9b55292de1ce492b9e347c66f4cf0111 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ffa5375dd2b431b852b9dd97c25d204" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 33711 } } peers { permanent_uuid: "9b55292de1ce492b9e347c66f4cf0111" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39435 } } peers { permanent_uuid: "7107058860f74ceab1ce49475d781b6c" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 38125 } }
I20260812 01:19:31.791539 5212 leader_election.cc:288] T d80c3945b82b43a6bfec8762aa7ef279 P 9b55292de1ce492b9e347c66f4cf0111 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ffa5375dd2b431b852b9dd97c25d204 (127.24.0.129:33711), 7107058860f74ceab1ce49475d781b6c (127.24.0.131:38125)
I20260812 01:19:31.791733 4636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d80c3945b82b43a6bfec8762aa7ef279" candidate_uuid: "9b55292de1ce492b9e347c66f4cf0111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ffa5375dd2b431b852b9dd97c25d204" is_pre_election: true
I20260812 01:19:31.791797 5069 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d80c3945b82b43a6bfec8762aa7ef279" candidate_uuid: "9b55292de1ce492b9e347c66f4cf0111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7107058860f74ceab1ce49475d781b6c" is_pre_election: true
I20260812 01:19:31.791855 4636 raft_consensus.cc:2454] T d80c3945b82b43a6bfec8762aa7ef279 P 0ffa5375dd2b431b852b9dd97c25d204 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9b55292de1ce492b9e347c66f4cf0111 in term 0.
I20260812 01:19:31.791906 5069 raft_consensus.cc:2454] T d80c3945b82b43a6bfec8762aa7ef279 P 7107058860f74ceab1ce49475d781b6c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9b55292de1ce492b9e347c66f4cf0111 in term 0.
I20260812 01:19:31.792030 4747 leader_election.cc:302] T d80c3945b82b43a6bfec8762aa7ef279 P 9b55292de1ce492b9e347c66f4cf0111 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ffa5375dd2b431b852b9dd97c25d204, 9b55292de1ce492b9e347c66f4cf0111; no voters:
I20260812 01:19:31.792181 5212 raft_consensus.cc:2781] T d80c3945b82b43a6bfec8762aa7ef279 P 9b55292de1ce492b9e347c66f4cf0111 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 01:19:31.792255 5212 raft_consensus.cc:482] T d80c3945b82b43a6bfec8762aa7ef279 P 9b55292de1ce492b9e347c66f4cf0111 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 01:19:31.792277 5212 raft_consensus.cc:3037] T d80c3945b82b43a6bfec8762aa7ef279 P 9b55292de1ce492b9e347c66f4cf0111 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:19:31.792948 5212 raft_consensus.cc:504] T d80c3945b82b43a6bfec8762aa7ef279 P 9b55292de1ce492b9e347c66f4cf0111 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ffa5375dd2b431b852b9dd97c25d204" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 33711 } } peers { permanent_uuid: "9b55292de1ce492b9e347c66f4cf0111" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39435 } } peers { permanent_uuid: "7107058860f74ceab1ce49475d781b6c" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 38125 } }
I20260812 01:19:31.793107 5212 leader_election.cc:288] T d80c3945b82b43a6bfec8762aa7ef279 P 9b55292de1ce492b9e347c66f4cf0111 [CANDIDATE]: Term 1 election: Requested vote from peers 0ffa5375dd2b431b852b9dd97c25d204 (127.24.0.129:33711), 7107058860f74ceab1ce49475d781b6c (127.24.0.131:38125)
I20260812 01:19:31.793247 4636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d80c3945b82b43a6bfec8762aa7ef279" candidate_uuid: "9b55292de1ce492b9e347c66f4cf0111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ffa5375dd2b431b852b9dd97c25d204"
I20260812 01:19:31.793269 5069 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d80c3945b82b43a6bfec8762aa7ef279" candidate_uuid: "9b55292de1ce492b9e347c66f4cf0111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7107058860f74ceab1ce49475d781b6c"
I20260812 01:19:31.793329 4636 raft_consensus.cc:3037] T d80c3945b82b43a6bfec8762aa7ef279 P 0ffa5375dd2b431b852b9dd97c25d204 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:19:31.793344 5069 raft_consensus.cc:3037] T d80c3945b82b43a6bfec8762aa7ef279 P 7107058860f74ceab1ce49475d781b6c [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:19:31.794128 4636 raft_consensus.cc:2454] T d80c3945b82b43a6bfec8762aa7ef279 P 0ffa5375dd2b431b852b9dd97c25d204 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b55292de1ce492b9e347c66f4cf0111 in term 1.
I20260812 01:19:31.794129 5069 raft_consensus.cc:2454] T d80c3945b82b43a6bfec8762aa7ef279 P 7107058860f74ceab1ce49475d781b6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b55292de1ce492b9e347c66f4cf0111 in term 1.
I20260812 01:19:31.794378 4748 leader_election.cc:302] T d80c3945b82b43a6bfec8762aa7ef279 P 9b55292de1ce492b9e347c66f4cf0111 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7107058860f74ceab1ce49475d781b6c, 9b55292de1ce492b9e347c66f4cf0111; no voters:
I20260812 01:19:31.794518 5212 raft_consensus.cc:2781] T d80c3945b82b43a6bfec8762aa7ef279 P 9b55292de1ce492b9e347c66f4cf0111 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:19:31.794589 5212 raft_consensus.cc:686] T d80c3945b82b43a6bfec8762aa7ef279 P 9b55292de1ce492b9e347c66f4cf0111 [term 1 LEADER]: Becoming Leader. State: Replica: 9b55292de1ce492b9e347c66f4cf0111, State: Running, Role: LEADER
I20260812 01:19:31.794625 5212 consensus_queue.cc:237] T d80c3945b82b43a6bfec8762aa7ef279 P 9b55292de1ce492b9e347c66f4cf0111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ffa5375dd2b431b852b9dd97c25d204" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 33711 } } peers { permanent_uuid: "9b55292de1ce492b9e347c66f4cf0111" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39435 } } peers { permanent_uuid: "7107058860f74ceab1ce49475d781b6c" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 38125 } }
I20260812 01:19:31.795222 4375 catalog_manager.cc:5495] T d80c3945b82b43a6bfec8762aa7ef279 P 9b55292de1ce492b9e347c66f4cf0111 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9b55292de1ce492b9e347c66f4cf0111 (127.24.0.130). New cstate: current_term: 1 leader_uuid: "9b55292de1ce492b9e347c66f4cf0111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ffa5375dd2b431b852b9dd97c25d204" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 33711 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9b55292de1ce492b9e347c66f4cf0111" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39435 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7107058860f74ceab1ce49475d781b6c" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 38125 } health_report { overall_health: UNKNOWN } } }
I20260812 01:19:31.802632 5170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:31.800559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40264 (local address 127.24.0.130:39435)
0812 01:19:31.800653 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:31.800655 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:31.800690 (+ 35us) server_negotiation.cc:407] Connection header received
0812 01:19:31.800758 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:31.800759 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:31.800772 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:31.800833 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:19:31.800940 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:31.801411 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:31.802110 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:31.802283 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:31.802333 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:31.802451 (+ 118us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:19:31.802477 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:19:31.802514 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:31.802565 (+ 51us) server_negotiation.cc:299] Negotiation successful
0812 01:19:31.802583 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260812 01:19:31.803342 5069 raft_consensus.cc:1261] T d80c3945b82b43a6bfec8762aa7ef279 P 7107058860f74ceab1ce49475d781b6c [term 1 FOLLOWER]: Refusing update from remote peer 9b55292de1ce492b9e347c66f4cf0111: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:19:31.803371 4636 raft_consensus.cc:1261] T d80c3945b82b43a6bfec8762aa7ef279 P 0ffa5375dd2b431b852b9dd97c25d204 [term 1 FOLLOWER]: Refusing update from remote peer 9b55292de1ce492b9e347c66f4cf0111: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:19:31.803571 5189 consensus_queue.cc:1035] T d80c3945b82b43a6bfec8762aa7ef279 P 9b55292de1ce492b9e347c66f4cf0111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7107058860f74ceab1ce49475d781b6c" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 38125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:19:31.803664 5212 consensus_queue.cc:1035] T d80c3945b82b43a6bfec8762aa7ef279 P 9b55292de1ce492b9e347c66f4cf0111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ffa5375dd2b431b852b9dd97c25d204" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 33711 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:19:31.806371 4372 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44788:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:19:31.806447 4372 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44788:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:19:31.807076 4372 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8cf578e52d544713ab6c35cb1333039f: Sending DeleteTablet for 3 replicas of tablet d80c3945b82b43a6bfec8762aa7ef279
I20260812 01:19:31.807358 4819 tablet_service.cc:1514] Processing DeleteTablet for tablet d80c3945b82b43a6bfec8762aa7ef279 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:19:31 UTC) from {username='oryx', principal='oryx/127.24.0.190@KRBTEST.COM'} at 127.0.0.1:40224
I20260812 01:19:31.807384 5036 tablet_service.cc:1514] Processing DeleteTablet for tablet d80c3945b82b43a6bfec8762aa7ef279 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:19:31 UTC) from {username='oryx', principal='oryx/127.24.0.190@KRBTEST.COM'} at 127.0.0.1:34842
I20260812 01:19:31.807453 5220 tablet_replica.cc:330] T d80c3945b82b43a6bfec8762aa7ef279 P 9b55292de1ce492b9e347c66f4cf0111: stopping tablet replica
I20260812 01:19:31.807479 5218 tablet_replica.cc:330] T d80c3945b82b43a6bfec8762aa7ef279 P 7107058860f74ceab1ce49475d781b6c: stopping tablet replica
I20260812 01:19:31.807507 5220 raft_consensus.cc:2229] T d80c3945b82b43a6bfec8762aa7ef279 P 9b55292de1ce492b9e347c66f4cf0111 [term 1 LEADER]: Raft consensus shutting down.
I20260812 01:19:31.807498 4605 tablet_service.cc:1514] Processing DeleteTablet for tablet d80c3945b82b43a6bfec8762aa7ef279 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:19:31 UTC) from {username='oryx', principal='oryx/127.24.0.190@KRBTEST.COM'} at 127.0.0.1:55444
I20260812 01:19:31.807549 5218 raft_consensus.cc:2229] T d80c3945b82b43a6bfec8762aa7ef279 P 7107058860f74ceab1ce49475d781b6c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 01:19:31.807574 5220 raft_consensus.cc:2258] T d80c3945b82b43a6bfec8762aa7ef279 P 9b55292de1ce492b9e347c66f4cf0111 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:19:31.807616 5218 raft_consensus.cc:2258] T d80c3945b82b43a6bfec8762aa7ef279 P 7107058860f74ceab1ce49475d781b6c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:19:31.807674 5221 tablet_replica.cc:330] T d80c3945b82b43a6bfec8762aa7ef279 P 0ffa5375dd2b431b852b9dd97c25d204: stopping tablet replica
I20260812 01:19:31.807732 5221 raft_consensus.cc:2229] T d80c3945b82b43a6bfec8762aa7ef279 P 0ffa5375dd2b431b852b9dd97c25d204 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 01:19:31.807775 5221 raft_consensus.cc:2258] T d80c3945b82b43a6bfec8762aa7ef279 P 0ffa5375dd2b431b852b9dd97c25d204 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:19:31.807931 5220 ts_tablet_manager.cc:1905] T d80c3945b82b43a6bfec8762aa7ef279 P 9b55292de1ce492b9e347c66f4cf0111: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:19:31.807860 4372 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44788:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 01:19:31.808013 4372 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 01:19:31.808012 5218 ts_tablet_manager.cc:1905] T d80c3945b82b43a6bfec8762aa7ef279 P 7107058860f74ceab1ce49475d781b6c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:19:31.808516 5221 ts_tablet_manager.cc:1905] T d80c3945b82b43a6bfec8762aa7ef279 P 0ffa5375dd2b431b852b9dd97c25d204: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:19:31.809176 5036 tablet_service.cc:1467] Processing CreateTablet for tablet 44fd9d1cc7b84935979d00f1e7768ce5 (DEFAULT_TABLE table=test-table [id=e210be09dd144afa9dfee06846ea3150]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:19:31.809271 5036 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44fd9d1cc7b84935979d00f1e7768ce5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:19:31.809636 5220 ts_tablet_manager.cc:1918] T d80c3945b82b43a6bfec8762aa7ef279 P 9b55292de1ce492b9e347c66f4cf0111: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 01:19:31.809688 5220 log.cc:1196] T d80c3945b82b43a6bfec8762aa7ef279 P 9b55292de1ce492b9e347c66f4cf0111: Deleting WAL directory at /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/minicluster-data/ts-1/wal/wals/d80c3945b82b43a6bfec8762aa7ef279
I20260812 01:19:31.809744 5218 ts_tablet_manager.cc:1918] T d80c3945b82b43a6bfec8762aa7ef279 P 7107058860f74ceab1ce49475d781b6c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 01:19:31.809803 5218 log.cc:1196] T d80c3945b82b43a6bfec8762aa7ef279 P 7107058860f74ceab1ce49475d781b6c: Deleting WAL directory at /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/minicluster-data/ts-2/wal/wals/d80c3945b82b43a6bfec8762aa7ef279
I20260812 01:19:31.809943 5220 ts_tablet_manager.cc:1939] T d80c3945b82b43a6bfec8762aa7ef279 P 9b55292de1ce492b9e347c66f4cf0111: Deleting consensus metadata
I20260812 01:19:31.810031 5218 ts_tablet_manager.cc:1939] T d80c3945b82b43a6bfec8762aa7ef279 P 7107058860f74ceab1ce49475d781b6c: Deleting consensus metadata
I20260812 01:19:31.810039 5221 ts_tablet_manager.cc:1918] T d80c3945b82b43a6bfec8762aa7ef279 P 0ffa5375dd2b431b852b9dd97c25d204: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 01:19:31.810102 5221 log.cc:1196] T d80c3945b82b43a6bfec8762aa7ef279 P 0ffa5375dd2b431b852b9dd97c25d204: Deleting WAL directory at /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/minicluster-data/ts-0/wal/wals/d80c3945b82b43a6bfec8762aa7ef279
I20260812 01:19:31.810316 5221 ts_tablet_manager.cc:1939] T d80c3945b82b43a6bfec8762aa7ef279 P 0ffa5375dd2b431b852b9dd97c25d204: Deleting consensus metadata
I20260812 01:19:31.810459 5172 tablet_bootstrap.cc:492] T 44fd9d1cc7b84935979d00f1e7768ce5 P 7107058860f74ceab1ce49475d781b6c: Bootstrap starting.
I20260812 01:19:31.810575 4356 catalog_manager.cc:4841] TS 7107058860f74ceab1ce49475d781b6c (127.24.0.131:38125): tablet d80c3945b82b43a6bfec8762aa7ef279 (table test-table [id=d943e1d35c574850bdbb2011a3c5576b]) successfully deleted
I20260812 01:19:31.810796 4355 catalog_manager.cc:4841] TS 0ffa5375dd2b431b852b9dd97c25d204 (127.24.0.129:33711): tablet d80c3945b82b43a6bfec8762aa7ef279 (table test-table [id=d943e1d35c574850bdbb2011a3c5576b]) successfully deleted
I20260812 01:19:31.810868 4355 catalog_manager.cc:4841] TS 9b55292de1ce492b9e347c66f4cf0111 (127.24.0.130:39435): tablet d80c3945b82b43a6bfec8762aa7ef279 (table test-table [id=d943e1d35c574850bdbb2011a3c5576b]) successfully deleted
I20260812 01:19:31.810891 4605 tablet_service.cc:1467] Processing CreateTablet for tablet 44fd9d1cc7b84935979d00f1e7768ce5 (DEFAULT_TABLE table=test-table [id=e210be09dd144afa9dfee06846ea3150]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:19:31.810891 4819 tablet_service.cc:1467] Processing CreateTablet for tablet 44fd9d1cc7b84935979d00f1e7768ce5 (DEFAULT_TABLE table=test-table [id=e210be09dd144afa9dfee06846ea3150]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:19:31.810995 4819 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44fd9d1cc7b84935979d00f1e7768ce5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:19:31.811009 4605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44fd9d1cc7b84935979d00f1e7768ce5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:19:31.811254 5172 tablet_bootstrap.cc:654] T 44fd9d1cc7b84935979d00f1e7768ce5 P 7107058860f74ceab1ce49475d781b6c: Neither blocks nor log segments found. Creating new log.
I20260812 01:19:31.812247 5179 tablet_bootstrap.cc:492] T 44fd9d1cc7b84935979d00f1e7768ce5 P 0ffa5375dd2b431b852b9dd97c25d204: Bootstrap starting.
I20260812 01:19:31.812551 5172 tablet_bootstrap.cc:492] T 44fd9d1cc7b84935979d00f1e7768ce5 P 7107058860f74ceab1ce49475d781b6c: No bootstrap required, opened a new log
I20260812 01:19:31.812611 5172 ts_tablet_manager.cc:1397] T 44fd9d1cc7b84935979d00f1e7768ce5 P 7107058860f74ceab1ce49475d781b6c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 01:19:31.812764 5172 raft_consensus.cc:348] T 44fd9d1cc7b84935979d00f1e7768ce5 P 7107058860f74ceab1ce49475d781b6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ffa5375dd2b431b852b9dd97c25d204" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 33711 } } peers { permanent_uuid: "9b55292de1ce492b9e347c66f4cf0111" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39435 } } peers { permanent_uuid: "7107058860f74ceab1ce49475d781b6c" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 38125 } }
I20260812 01:19:31.812835 5172 raft_consensus.cc:374] T 44fd9d1cc7b84935979d00f1e7768ce5 P 7107058860f74ceab1ce49475d781b6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:19:31.812860 5172 raft_consensus.cc:729] T 44fd9d1cc7b84935979d00f1e7768ce5 P 7107058860f74ceab1ce49475d781b6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7107058860f74ceab1ce49475d781b6c, State: Initialized, Role: FOLLOWER
I20260812 01:19:31.812892 5179 tablet_bootstrap.cc:654] T 44fd9d1cc7b84935979d00f1e7768ce5 P 0ffa5375dd2b431b852b9dd97c25d204: Neither blocks nor log segments found. Creating new log.
I20260812 01:19:31.812911 5172 consensus_queue.cc:260] T 44fd9d1cc7b84935979d00f1e7768ce5 P 7107058860f74ceab1ce49475d781b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ffa5375dd2b431b852b9dd97c25d204" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 33711 } } peers { permanent_uuid: "9b55292de1ce492b9e347c66f4cf0111" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39435 } } peers { permanent_uuid: "7107058860f74ceab1ce49475d781b6c" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 38125 } }
I20260812 01:19:31.813030 5172 ts_tablet_manager.cc:1428] T 44fd9d1cc7b84935979d00f1e7768ce5 P 7107058860f74ceab1ce49475d781b6c: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 01:19:31.813504 5173 tablet_bootstrap.cc:492] T 44fd9d1cc7b84935979d00f1e7768ce5 P 9b55292de1ce492b9e347c66f4cf0111: Bootstrap starting.
I20260812 01:19:31.813645 5179 tablet_bootstrap.cc:492] T 44fd9d1cc7b84935979d00f1e7768ce5 P 0ffa5375dd2b431b852b9dd97c25d204: No bootstrap required, opened a new log
I20260812 01:19:31.813688 5179 ts_tablet_manager.cc:1397] T 44fd9d1cc7b84935979d00f1e7768ce5 P 0ffa5375dd2b431b852b9dd97c25d204: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260812 01:19:31.813794 5179 raft_consensus.cc:348] T 44fd9d1cc7b84935979d00f1e7768ce5 P 0ffa5375dd2b431b852b9dd97c25d204 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ffa5375dd2b431b852b9dd97c25d204" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 33711 } } peers { permanent_uuid: "9b55292de1ce492b9e347c66f4cf0111" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39435 } } peers { permanent_uuid: "7107058860f74ceab1ce49475d781b6c" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 38125 } }
I20260812 01:19:31.813808 5173 tablet_bootstrap.cc:654] T 44fd9d1cc7b84935979d00f1e7768ce5 P 9b55292de1ce492b9e347c66f4cf0111: Neither blocks nor log segments found. Creating new log.
I20260812 01:19:31.813835 5179 raft_consensus.cc:374] T 44fd9d1cc7b84935979d00f1e7768ce5 P 0ffa5375dd2b431b852b9dd97c25d204 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:19:31.813849 5179 raft_consensus.cc:729] T 44fd9d1cc7b84935979d00f1e7768ce5 P 0ffa5375dd2b431b852b9dd97c25d204 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ffa5375dd2b431b852b9dd97c25d204, State: Initialized, Role: FOLLOWER
I20260812 01:19:31.813906 5179 consensus_queue.cc:260] T 44fd9d1cc7b84935979d00f1e7768ce5 P 0ffa5375dd2b431b852b9dd97c25d204 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ffa5375dd2b431b852b9dd97c25d204" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 33711 } } peers { permanent_uuid: "9b55292de1ce492b9e347c66f4cf0111" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39435 } } peers { permanent_uuid: "7107058860f74ceab1ce49475d781b6c" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 38125 } }
I20260812 01:19:31.813994 5179 ts_tablet_manager.cc:1428] T 44fd9d1cc7b84935979d00f1e7768ce5 P 0ffa5375dd2b431b852b9dd97c25d204: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:31.814476 5173 tablet_bootstrap.cc:492] T 44fd9d1cc7b84935979d00f1e7768ce5 P 9b55292de1ce492b9e347c66f4cf0111: No bootstrap required, opened a new log
I20260812 01:19:31.814523 5173 ts_tablet_manager.cc:1397] T 44fd9d1cc7b84935979d00f1e7768ce5 P 9b55292de1ce492b9e347c66f4cf0111: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260812 01:19:31.814630 5173 raft_consensus.cc:348] T 44fd9d1cc7b84935979d00f1e7768ce5 P 9b55292de1ce492b9e347c66f4cf0111 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ffa5375dd2b431b852b9dd97c25d204" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 33711 } } peers { permanent_uuid: "9b55292de1ce492b9e347c66f4cf0111" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39435 } } peers { permanent_uuid: "7107058860f74ceab1ce49475d781b6c" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 38125 } }
I20260812 01:19:31.814692 5173 raft_consensus.cc:374] T 44fd9d1cc7b84935979d00f1e7768ce5 P 9b55292de1ce492b9e347c66f4cf0111 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:19:31.814716 5173 raft_consensus.cc:729] T 44fd9d1cc7b84935979d00f1e7768ce5 P 9b55292de1ce492b9e347c66f4cf0111 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b55292de1ce492b9e347c66f4cf0111, State: Initialized, Role: FOLLOWER
I20260812 01:19:31.814751 5173 consensus_queue.cc:260] T 44fd9d1cc7b84935979d00f1e7768ce5 P 9b55292de1ce492b9e347c66f4cf0111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ffa5375dd2b431b852b9dd97c25d204" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 33711 } } peers { permanent_uuid: "9b55292de1ce492b9e347c66f4cf0111" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39435 } } peers { permanent_uuid: "7107058860f74ceab1ce49475d781b6c" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 38125 } }
I20260812 01:19:31.814833 5173 ts_tablet_manager.cc:1428] T 44fd9d1cc7b84935979d00f1e7768ce5 P 9b55292de1ce492b9e347c66f4cf0111: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:31.935015 5185 raft_consensus.cc:482] T 44fd9d1cc7b84935979d00f1e7768ce5 P 0ffa5375dd2b431b852b9dd97c25d204 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 01:19:31.935124 5185 raft_consensus.cc:504] T 44fd9d1cc7b84935979d00f1e7768ce5 P 0ffa5375dd2b431b852b9dd97c25d204 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ffa5375dd2b431b852b9dd97c25d204" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 33711 } } peers { permanent_uuid: "9b55292de1ce492b9e347c66f4cf0111" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39435 } } peers { permanent_uuid: "7107058860f74ceab1ce49475d781b6c" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 38125 } }
I20260812 01:19:31.935441 5185 leader_election.cc:288] T 44fd9d1cc7b84935979d00f1e7768ce5 P 0ffa5375dd2b431b852b9dd97c25d204 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9b55292de1ce492b9e347c66f4cf0111 (127.24.0.130:39435), 7107058860f74ceab1ce49475d781b6c (127.24.0.131:38125)
I20260812 01:19:31.938606 5170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:31.935714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.129:50787 (local address 127.24.0.130:39435)
0812 01:19:31.935804 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:31.935806 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:31.935983 (+ 177us) server_negotiation.cc:407] Connection header received
0812 01:19:31.936020 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:31.936021 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:31.936034 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:31.936140 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:19:31.936334 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:31.936850 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:31.937867 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:31.938239 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:31.938299 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:31.938305 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:31.938481 (+ 176us) server_negotiation.cc:299] Negotiation successful
0812 01:19:31.938529 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 01:19:31.938613 5291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:31.935631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.130:39435 (local address 127.24.0.129:50787)
0812 01:19:31.935867 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:31.935878 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:31.935959 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:31.936150 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:31.936150 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:31.936154 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:19:31.936257 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:31.936260 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:31.936934 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:31.936935 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:31.937774 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:31.937776 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:31.938324 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:31.938325 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:31.938404 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:31.938409 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 01:19:31.938462 (+ 53us) client_negotiation.cc:240] Negotiation successful
0812 01:19:31.938504 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":68,"threads_started":1}
I20260812 01:19:31.938726 5293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:31.935713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.129:49471 (local address 127.24.0.131:38125)
0812 01:19:31.935860 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:31.935862 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:31.936105 (+ 243us) server_negotiation.cc:407] Connection header received
0812 01:19:31.936182 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:31.936183 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:31.936194 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:31.936259 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:19:31.936453 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:31.936955 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:31.937911 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:31.938301 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:31.938378 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:31.938387 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:31.938586 (+ 199us) server_negotiation.cc:299] Negotiation successful
0812 01:19:31.938635 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":73,"threads_started":1}
I20260812 01:19:31.938968 5292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:31.935631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.131:38125 (local address 127.24.0.129:49471)
0812 01:19:31.936002 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:31.936013 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:31.936109 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:31.936269 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:31.936270 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:31.936272 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:19:31.936377 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:31.936379 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:31.937038 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:31.937039 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:31.937789 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:31.937791 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:31.938402 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:31.938403 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:31.938497 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:31.938502 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 01:19:31.938560 (+ 58us) client_negotiation.cc:240] Negotiation successful
0812 01:19:31.938605 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":349,"mutex_wait_us":3,"thread_start_us":69,"threads_started":1}
I20260812 01:19:31.939226 4858 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44fd9d1cc7b84935979d00f1e7768ce5" candidate_uuid: "0ffa5375dd2b431b852b9dd97c25d204" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b55292de1ce492b9e347c66f4cf0111" is_pre_election: true
I20260812 01:19:31.939309 5069 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44fd9d1cc7b84935979d00f1e7768ce5" candidate_uuid: "0ffa5375dd2b431b852b9dd97c25d204" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7107058860f74ceab1ce49475d781b6c" is_pre_election: true
I20260812 01:19:31.939374 4858 raft_consensus.cc:2454] T 44fd9d1cc7b84935979d00f1e7768ce5 P 9b55292de1ce492b9e347c66f4cf0111 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ffa5375dd2b431b852b9dd97c25d204 in term 0.
I20260812 01:19:31.939415 5069 raft_consensus.cc:2454] T 44fd9d1cc7b84935979d00f1e7768ce5 P 7107058860f74ceab1ce49475d781b6c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ffa5375dd2b431b852b9dd97c25d204 in term 0.
I20260812 01:19:31.939589 4519 leader_election.cc:302] T 44fd9d1cc7b84935979d00f1e7768ce5 P 0ffa5375dd2b431b852b9dd97c25d204 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ffa5375dd2b431b852b9dd97c25d204, 9b55292de1ce492b9e347c66f4cf0111; no voters:
I20260812 01:19:31.939731 5185 raft_consensus.cc:2781] T 44fd9d1cc7b84935979d00f1e7768ce5 P 0ffa5375dd2b431b852b9dd97c25d204 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 01:19:31.939774 5185 raft_consensus.cc:482] T 44fd9d1cc7b84935979d00f1e7768ce5 P 0ffa5375dd2b431b852b9dd97c25d204 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 01:19:31.939800 5185 raft_consensus.cc:3037] T 44fd9d1cc7b84935979d00f1e7768ce5 P 0ffa5375dd2b431b852b9dd97c25d204 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:19:31.940601 5185 raft_consensus.cc:504] T 44fd9d1cc7b84935979d00f1e7768ce5 P 0ffa5375dd2b431b852b9dd97c25d204 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ffa5375dd2b431b852b9dd97c25d204" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 33711 } } peers { permanent_uuid: "9b55292de1ce492b9e347c66f4cf0111" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39435 } } peers { permanent_uuid: "7107058860f74ceab1ce49475d781b6c" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 38125 } }
I20260812 01:19:31.940753 5185 leader_election.cc:288] T 44fd9d1cc7b84935979d00f1e7768ce5 P 0ffa5375dd2b431b852b9dd97c25d204 [CANDIDATE]: Term 1 election: Requested vote from peers 9b55292de1ce492b9e347c66f4cf0111 (127.24.0.130:39435), 7107058860f74ceab1ce49475d781b6c (127.24.0.131:38125)
I20260812 01:19:31.940933 4858 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44fd9d1cc7b84935979d00f1e7768ce5" candidate_uuid: "0ffa5375dd2b431b852b9dd97c25d204" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b55292de1ce492b9e347c66f4cf0111"
I20260812 01:19:31.941005 4858 raft_consensus.cc:3037] T 44fd9d1cc7b84935979d00f1e7768ce5 P 9b55292de1ce492b9e347c66f4cf0111 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:19:31.940937 5069 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44fd9d1cc7b84935979d00f1e7768ce5" candidate_uuid: "0ffa5375dd2b431b852b9dd97c25d204" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7107058860f74ceab1ce49475d781b6c"
I20260812 01:19:31.941037 5069 raft_consensus.cc:3037] T 44fd9d1cc7b84935979d00f1e7768ce5 P 7107058860f74ceab1ce49475d781b6c [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:19:31.941671 5069 raft_consensus.cc:2454] T 44fd9d1cc7b84935979d00f1e7768ce5 P 7107058860f74ceab1ce49475d781b6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ffa5375dd2b431b852b9dd97c25d204 in term 1.
I20260812 01:19:31.941673 4858 raft_consensus.cc:2454] T 44fd9d1cc7b84935979d00f1e7768ce5 P 9b55292de1ce492b9e347c66f4cf0111 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ffa5375dd2b431b852b9dd97c25d204 in term 1.
I20260812 01:19:31.941864 4519 leader_election.cc:302] T 44fd9d1cc7b84935979d00f1e7768ce5 P 0ffa5375dd2b431b852b9dd97c25d204 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ffa5375dd2b431b852b9dd97c25d204, 9b55292de1ce492b9e347c66f4cf0111; no voters:
I20260812 01:19:31.941969 5185 raft_consensus.cc:2781] T 44fd9d1cc7b84935979d00f1e7768ce5 P 0ffa5375dd2b431b852b9dd97c25d204 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:19:31.942121 5185 raft_consensus.cc:686] T 44fd9d1cc7b84935979d00f1e7768ce5 P 0ffa5375dd2b431b852b9dd97c25d204 [term 1 LEADER]: Becoming Leader. State: Replica: 0ffa5375dd2b431b852b9dd97c25d204, State: Running, Role: LEADER
I20260812 01:19:31.942207 5185 consensus_queue.cc:237] T 44fd9d1cc7b84935979d00f1e7768ce5 P 0ffa5375dd2b431b852b9dd97c25d204 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ffa5375dd2b431b852b9dd97c25d204" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 33711 } } peers { permanent_uuid: "9b55292de1ce492b9e347c66f4cf0111" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39435 } } peers { permanent_uuid: "7107058860f74ceab1ce49475d781b6c" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 38125 } }
I20260812 01:19:31.942927 4375 catalog_manager.cc:5495] T 44fd9d1cc7b84935979d00f1e7768ce5 P 0ffa5375dd2b431b852b9dd97c25d204 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ffa5375dd2b431b852b9dd97c25d204 (127.24.0.129). New cstate: current_term: 1 leader_uuid: "0ffa5375dd2b431b852b9dd97c25d204" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ffa5375dd2b431b852b9dd97c25d204" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 33711 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9b55292de1ce492b9e347c66f4cf0111" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39435 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7107058860f74ceab1ce49475d781b6c" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 38125 } health_report { overall_health: UNKNOWN } } }
I20260812 01:19:31.966846 5298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:31.964562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55458 (local address 127.24.0.129:33711)
0812 01:19:31.964734 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:31.964736 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:31.964743 (+ 7us) server_negotiation.cc:407] Connection header received
0812 01:19:31.964775 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:31.964776 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:31.964794 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:31.964864 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:19:31.964967 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:31.965455 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:31.966147 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:31.966335 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:31.966385 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:31.966499 (+ 114us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:19:31.966614 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:19:31.966665 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:31.966712 (+ 47us) server_negotiation.cc:299] Negotiation successful
0812 01:19:31.966727 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":115,"threads_started":1}
I20260812 01:19:31.971570 5170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:31.969137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.129:57073 (local address 127.24.0.130:39435)
0812 01:19:31.969217 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:31.969220 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:31.969324 (+ 104us) server_negotiation.cc:407] Connection header received
0812 01:19:31.969388 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:31.969389 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:31.969400 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:31.969456 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:19:31.969608 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:31.970072 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:31.970850 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:31.971058 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:31.971144 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:31.971323 (+ 179us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:19:31.971357 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:19:31.971427 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:31.971484 (+ 57us) server_negotiation.cc:299] Negotiation successful
0812 01:19:31.971503 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 01:19:31.971611 5303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:31.969046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.130:39435 (local address 127.24.0.129:57073)
0812 01:19:31.969246 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:31.969260 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:31.969333 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:31.969460 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:31.969461 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:31.969462 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 01:19:31.969537 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:31.969538 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:31.970154 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:31.970155 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:31.970757 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:31.970759 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:31.971150 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:31.971151 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:31.971224 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:31.971237 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 01:19:31.971430 (+ 193us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 01:19:31.971433 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 01:19:31.971481 (+ 48us) client_negotiation.cc:240] Negotiation successful
0812 01:19:31.971527 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":100,"threads_started":1}
I20260812 01:19:31.973912 5069 raft_consensus.cc:1261] T 44fd9d1cc7b84935979d00f1e7768ce5 P 7107058860f74ceab1ce49475d781b6c [term 1 FOLLOWER]: Refusing update from remote peer 0ffa5375dd2b431b852b9dd97c25d204: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:19:31.974030 4858 raft_consensus.cc:1261] T 44fd9d1cc7b84935979d00f1e7768ce5 P 9b55292de1ce492b9e347c66f4cf0111 [term 1 FOLLOWER]: Refusing update from remote peer 0ffa5375dd2b431b852b9dd97c25d204: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:19:31.974195 5295 consensus_queue.cc:1035] T 44fd9d1cc7b84935979d00f1e7768ce5 P 0ffa5375dd2b431b852b9dd97c25d204 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7107058860f74ceab1ce49475d781b6c" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 38125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:19:31.974305 5185 consensus_queue.cc:1035] T 44fd9d1cc7b84935979d00f1e7768ce5 P 0ffa5375dd2b431b852b9dd97c25d204 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b55292de1ce492b9e347c66f4cf0111" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39435 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:19:31.981566 5298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:31.979422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.130:34631 (local address 127.24.0.129:33711)
0812 01:19:31.979457 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:31.979459 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:31.979466 (+ 7us) server_negotiation.cc:407] Connection header received
0812 01:19:31.979487 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:31.979488 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:31.979499 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:31.979581 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:19:31.979673 (+ 92us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:31.980195 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:31.980904 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:31.981143 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:31.981211 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:31.981352 (+ 141us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:19:31.981382 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:19:31.981437 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:31.981486 (+ 49us) server_negotiation.cc:299] Negotiation successful
0812 01:19:31.981505 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 01:19:31.981596 5258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:31.979255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.129:33711 (local address 127.24.0.130:34631)
0812 01:19:31.979305 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:31.979314 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:31.979348 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:31.979581 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:31.979582 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:31.979583 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 01:19:31.979622 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:31.979624 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:31.980272 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:31.980273 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:31.980805 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:31.980807 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:31.981209 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:31.981210 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:31.981265 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:31.981272 (+ 7us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 01:19:31.981440 (+ 168us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 01:19:31.981442 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 01:19:31.981485 (+ 43us) client_negotiation.cc:240] Negotiation successful
0812 01:19:31.981530 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260812 01:19:31.987689 4372 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44788:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:19:31.987787 4372 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44788:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:19:31.988442 4372 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8cf578e52d544713ab6c35cb1333039f: Sending DeleteTablet for 3 replicas of tablet 44fd9d1cc7b84935979d00f1e7768ce5
I20260812 01:19:31.988684 4605 tablet_service.cc:1514] Processing DeleteTablet for tablet 44fd9d1cc7b84935979d00f1e7768ce5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:19:31 UTC) from {username='oryx', principal='oryx/127.24.0.190@KRBTEST.COM'} at 127.0.0.1:55444
I20260812 01:19:31.988791 5036 tablet_service.cc:1514] Processing DeleteTablet for tablet 44fd9d1cc7b84935979d00f1e7768ce5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:19:31 UTC) from {username='oryx', principal='oryx/127.24.0.190@KRBTEST.COM'} at 127.0.0.1:34842
I20260812 01:19:31.988884 5221 tablet_replica.cc:330] T 44fd9d1cc7b84935979d00f1e7768ce5 P 0ffa5375dd2b431b852b9dd97c25d204: stopping tablet replica
I20260812 01:19:31.988940 5221 raft_consensus.cc:2229] T 44fd9d1cc7b84935979d00f1e7768ce5 P 0ffa5375dd2b431b852b9dd97c25d204 [term 1 LEADER]: Raft consensus shutting down.
I20260812 01:19:31.989019 5221 raft_consensus.cc:2258] T 44fd9d1cc7b84935979d00f1e7768ce5 P 0ffa5375dd2b431b852b9dd97c25d204 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:19:31.991120 5218 tablet_replica.cc:330] T 44fd9d1cc7b84935979d00f1e7768ce5 P 7107058860f74ceab1ce49475d781b6c: stopping tablet replica
I20260812 01:19:31.991204 5218 raft_consensus.cc:2229] T 44fd9d1cc7b84935979d00f1e7768ce5 P 7107058860f74ceab1ce49475d781b6c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 01:19:31.991250 5218 raft_consensus.cc:2258] T 44fd9d1cc7b84935979d00f1e7768ce5 P 7107058860f74ceab1ce49475d781b6c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:19:31.991533 5221 ts_tablet_manager.cc:1905] T 44fd9d1cc7b84935979d00f1e7768ce5 P 0ffa5375dd2b431b852b9dd97c25d204: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:19:31.991776 5218 ts_tablet_manager.cc:1905] T 44fd9d1cc7b84935979d00f1e7768ce5 P 7107058860f74ceab1ce49475d781b6c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:19:31.991798 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 4497
I20260812 01:19:31.993749 5218 ts_tablet_manager.cc:1918] T 44fd9d1cc7b84935979d00f1e7768ce5 P 7107058860f74ceab1ce49475d781b6c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 01:19:31.993810 5218 log.cc:1196] T 44fd9d1cc7b84935979d00f1e7768ce5 P 7107058860f74ceab1ce49475d781b6c: Deleting WAL directory at /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/minicluster-data/ts-2/wal/wals/44fd9d1cc7b84935979d00f1e7768ce5
I20260812 01:19:31.994057 5218 ts_tablet_manager.cc:1939] T 44fd9d1cc7b84935979d00f1e7768ce5 P 7107058860f74ceab1ce49475d781b6c: Deleting consensus metadata
I20260812 01:19:31.994437 4356 catalog_manager.cc:4841] TS 7107058860f74ceab1ce49475d781b6c (127.24.0.131:38125): tablet 44fd9d1cc7b84935979d00f1e7768ce5 (table test-table [id=e210be09dd144afa9dfee06846ea3150]) successfully deleted
I20260812 01:19:31.995730 4819 tablet_service.cc:1514] Processing DeleteTablet for tablet 44fd9d1cc7b84935979d00f1e7768ce5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:19:31 UTC) from {username='oryx', principal='oryx/127.24.0.190@KRBTEST.COM'} at 127.0.0.1:40224
I20260812 01:19:31.996114 5220 tablet_replica.cc:330] T 44fd9d1cc7b84935979d00f1e7768ce5 P 9b55292de1ce492b9e347c66f4cf0111: stopping tablet replica
I20260812 01:19:31.996207 5220 raft_consensus.cc:2229] T 44fd9d1cc7b84935979d00f1e7768ce5 P 9b55292de1ce492b9e347c66f4cf0111 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 01:19:31.996243 5220 raft_consensus.cc:2258] T 44fd9d1cc7b84935979d00f1e7768ce5 P 9b55292de1ce492b9e347c66f4cf0111 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:19:31.998224 5220 ts_tablet_manager.cc:1905] T 44fd9d1cc7b84935979d00f1e7768ce5 P 9b55292de1ce492b9e347c66f4cf0111: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:19:32.000319 5220 ts_tablet_manager.cc:1918] T 44fd9d1cc7b84935979d00f1e7768ce5 P 9b55292de1ce492b9e347c66f4cf0111: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 01:19:32.000409 5220 log.cc:1196] T 44fd9d1cc7b84935979d00f1e7768ce5 P 9b55292de1ce492b9e347c66f4cf0111: Deleting WAL directory at /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786497534195564-24578-0/minicluster-data/ts-1/wal/wals/44fd9d1cc7b84935979d00f1e7768ce5
I20260812 01:19:32.000618 5220 ts_tablet_manager.cc:1939] T 44fd9d1cc7b84935979d00f1e7768ce5 P 9b55292de1ce492b9e347c66f4cf0111: Deleting consensus metadata
I20260812 01:19:32.000958 4355 catalog_manager.cc:4841] TS 9b55292de1ce492b9e347c66f4cf0111 (127.24.0.130:39435): tablet 44fd9d1cc7b84935979d00f1e7768ce5 (table test-table [id=e210be09dd144afa9dfee06846ea3150]) successfully deleted
W20260812 01:19:32.005203 4355 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.0.129:33711 (error 108)
I20260812 01:19:32.005376 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 4718
I20260812 01:19:32.011901 5309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:32.011645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.129:33711 (local address 127.0.0.1:55470)
0812 01:19:32.011769 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:32.011836 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.129:33711: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":74,"threads_started":1}
W20260812 01:19:32.013585 4355 catalog_manager.cc:4584] TS 0ffa5375dd2b431b852b9dd97c25d204 (127.24.0.129:33711): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 44fd9d1cc7b84935979d00f1e7768ce5: Network error: Client connection negotiation failed: client connection to 127.24.0.129:33711: connect: Connection refused (error 111)
I20260812 01:19:32.020545 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 4931
I20260812 01:19:32.037101 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 4322
2026-08-12T01:19:32Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3129 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786497534195564-24578-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 01:19:32 dist-test-slave-t3q3 krb5kdc[5312](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 01:19:32 dist-test-slave-t3q3 krb5kdc[5312](info): set up 2 sockets
Aug 12 01:19:32 dist-test-slave-t3q3 krb5kdc[5312](info): commencing operation
krb5kdc: starting...
W20260812 01:19:33.321504 24578 mini_kdc.cc:121] Time spent starting KDC: real 1.259s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786497534195564-24578-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786497534195564-24578-0/krb5kdc/test-user.keytab.
Aug 12 01:19:33 dist-test-slave-t3q3 krb5kdc[5312](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497573, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T01:19:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:19:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.0.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.0.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.0.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab.
Entry for principal kudu/127.24.0.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab.
I20260812 01:19:33.450886 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786497534195564-24578-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786497534195564-24578-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786497534195564-24578-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.190:39611
--webserver_interface=127.24.0.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab
--principal=kudu/127.24.0.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:40191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.0.190:39611
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:19:33.534139 5331 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:19:33.534399 5331 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:19:33.534438 5331 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:19:33.535584 5331 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:19:33.535651 5331 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:19:33.535679 5331 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:19:33.535701 5331 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:19:33.535727 5331 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:19:33.537279 5331 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:40191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786497534195564-24578-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786497534195564-24578-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.0.190:39611
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.190:39611
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.0.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5331
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786497534195564-24578-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:19:33.537580 5331 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:19:33.537817 5331 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:19:33.542949 5348 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:33.542939 5341 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:33.542939 5344 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:19:33.543241 5331 server_base.cc:1034] running on GCE node
I20260812 01:19:33.543546 5331 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:19:33.543876 5331 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:19:33.545042 5331 hybrid_clock.cc:648] HybridClock initialized: now 1786497573545017 us; error 39 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-12T01:19:33Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1516 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 01:19:33 dist-test-slave-t3q3 krb5kdc[5353](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 01:19:33 dist-test-slave-t3q3 krb5kdc[5353](info): set up 2 sockets
Aug 12 01:19:33 dist-test-slave-t3q3 krb5kdc[5353](info): commencing operation
krb5kdc: starting...
W20260812 01:19:34.828665 24578 mini_kdc.cc:121] Time spent starting KDC: real 1.249s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/krb5kdc/test-user.keytab.
Aug 12 01:19:34 dist-test-slave-t3q3 krb5kdc[5353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497574, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T01:19:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:19:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.0.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.0.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.0.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab.
Entry for principal kudu/127.24.0.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab.
I20260812 01:19:34.959514 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.190:43253
--webserver_interface=127.24.0.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab
--principal=kudu/127.24.0.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:39433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.0.190:43253
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:19:35.046029 5378 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:19:35.046249 5378 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:19:35.046274 5378 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:19:35.047366 5378 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:19:35.047401 5378 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:19:35.047415 5378 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:19:35.047428 5378 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:19:35.047441 5378 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:19:35.048940 5378 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:39433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.0.190:43253
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab
--principal=kudu/127.24.0.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.190:43253
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.0.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5378
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:19:35.049151 5378 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:19:35.049336 5378 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:19:35.054389 5390 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:35.054378 5385 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:35.054350 5386 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:19:35.054435 5378 server_base.cc:1034] running on GCE node
I20260812 01:19:35.054874 5378 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:19:35.055183 5378 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:19:35.056358 5378 hybrid_clock.cc:648] HybridClock initialized: now 1786497575056333 us; error 36 us; skew 500 ppm
Aug 12 01:19:35 dist-test-slave-t3q3 krb5kdc[5353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497575, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:19:35.058933 5378 init.cc:377] Logged in from keytab as kudu/127.24.0.190@KRBTEST.COM (short username kudu)
I20260812 01:19:35.059379 5378 webserver.cc:460] Webserver started at http://127.24.0.190:37749/ using document root <none> and password file <none>
I20260812 01:19:35.059624 5378 fs_manager.cc:359] Metadata directory not provided
I20260812 01:19:35.059696 5378 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:19:35.059849 5378 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:19:35.060979 5378 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/master-0/data/instance:
uuid: "4e335b200cc4457d9e4f977a36acb97e"
format_stamp: "Formatted at 2026-08-12 01:19:35 on dist-test-slave-t3q3"
server_key: "a7c8ca49d649b269cede4633e5a91e5c"
server_key_iv: "c2e8f993c572066cd475b0142569251b"
server_key_version: "encryptionkey@0"
I20260812 01:19:35.061357 5378 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/master-0/wal/instance:
uuid: "4e335b200cc4457d9e4f977a36acb97e"
format_stamp: "Formatted at 2026-08-12 01:19:35 on dist-test-slave-t3q3"
server_key: "a7c8ca49d649b269cede4633e5a91e5c"
server_key_iv: "c2e8f993c572066cd475b0142569251b"
server_key_version: "encryptionkey@0"
I20260812 01:19:35.062806 5378 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 01:19:35.063534 5402 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:35.063711 5378 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 01:19:35.063822 5378 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/master-0/wal
uuid: "4e335b200cc4457d9e4f977a36acb97e"
format_stamp: "Formatted at 2026-08-12 01:19:35 on dist-test-slave-t3q3"
server_key: "a7c8ca49d649b269cede4633e5a91e5c"
server_key_iv: "c2e8f993c572066cd475b0142569251b"
server_key_version: "encryptionkey@0"
I20260812 01:19:35.063902 5378 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:19:35.076525 5378 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:19:35.078999 5378 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:19:35.079180 5378 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:19:35.083552 5378 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.190:43253
I20260812 01:19:35.083590 5487 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.190:43253 every 8 connection(s)
I20260812 01:19:35.083976 5378 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
I20260812 01:19:35.084965 5488 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:19:35.086191 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 5378
I20260812 01:19:35.086342 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/master-0/wal/instance
I20260812 01:19:35.086546 24578 external_mini_cluster.cc:1428] Setting key 8de2e063fc639843e4f46c19cf833476
I20260812 01:19:35.088030 5488 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e335b200cc4457d9e4f977a36acb97e: Bootstrap starting.
I20260812 01:19:35.088753 5488 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4e335b200cc4457d9e4f977a36acb97e: Neither blocks nor log segments found. Creating new log.
I20260812 01:19:35.089093 5488 log.cc:824] T 00000000000000000000000000000000 P 4e335b200cc4457d9e4f977a36acb97e: Log is configured to *not* fsync() on all Append() calls
I20260812 01:19:35.089915 5488 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e335b200cc4457d9e4f977a36acb97e: No bootstrap required, opened a new log
I20260812 01:19:35.092206 5488 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4e335b200cc4457d9e4f977a36acb97e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e335b200cc4457d9e4f977a36acb97e" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 43253 } }
I20260812 01:19:35.092362 5488 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4e335b200cc4457d9e4f977a36acb97e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:19:35.092409 5488 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4e335b200cc4457d9e4f977a36acb97e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e335b200cc4457d9e4f977a36acb97e, State: Initialized, Role: FOLLOWER
I20260812 01:19:35.092547 5488 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4e335b200cc4457d9e4f977a36acb97e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e335b200cc4457d9e4f977a36acb97e" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 43253 } }
I20260812 01:19:35.092648 5488 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4e335b200cc4457d9e4f977a36acb97e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:19:35.092701 5488 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4e335b200cc4457d9e4f977a36acb97e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:19:35.092754 5488 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4e335b200cc4457d9e4f977a36acb97e [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:19:35.093320 5488 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4e335b200cc4457d9e4f977a36acb97e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e335b200cc4457d9e4f977a36acb97e" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 43253 } }
I20260812 01:19:35.093443 5488 leader_election.cc:302] T 00000000000000000000000000000000 P 4e335b200cc4457d9e4f977a36acb97e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4e335b200cc4457d9e4f977a36acb97e; no voters:
I20260812 01:19:35.093626 5488 leader_election.cc:288] T 00000000000000000000000000000000 P 4e335b200cc4457d9e4f977a36acb97e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 01:19:35.093660 5495 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4e335b200cc4457d9e4f977a36acb97e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:19:35.093854 5495 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4e335b200cc4457d9e4f977a36acb97e [term 1 LEADER]: Becoming Leader. State: Replica: 4e335b200cc4457d9e4f977a36acb97e, State: Running, Role: LEADER
I20260812 01:19:35.093940 5488 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4e335b200cc4457d9e4f977a36acb97e [sys.catalog]: configured and running, proceeding with master startup.
Aug 12 01:19:35 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497574, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
I20260812 01:19:35.094063 5495 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4e335b200cc4457d9e4f977a36acb97e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e335b200cc4457d9e4f977a36acb97e" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 43253 } }
I20260812 01:19:35.094439 5497 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e335b200cc4457d9e4f977a36acb97e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4e335b200cc4457d9e4f977a36acb97e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e335b200cc4457d9e4f977a36acb97e" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 43253 } } }
I20260812 01:19:35.094421 5495 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e335b200cc4457d9e4f977a36acb97e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4e335b200cc4457d9e4f977a36acb97e. Latest consensus state: current_term: 1 leader_uuid: "4e335b200cc4457d9e4f977a36acb97e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e335b200cc4457d9e4f977a36acb97e" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 43253 } } }
I20260812 01:19:35.094539 5497 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e335b200cc4457d9e4f977a36acb97e [sys.catalog]: This master's current role is: LEADER
I20260812 01:19:35.094545 5495 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e335b200cc4457d9e4f977a36acb97e [sys.catalog]: This master's current role is: LEADER
I20260812 01:19:35.094769 5502 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:19:35.095341 5502 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:19:35.097137 5502 catalog_manager.cc:1347] Generated new cluster ID: ede4c36ad3b145609654e79cb5dd8dae
I20260812 01:19:35.097189 5502 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 01:19:35.098486 5493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:35.087148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41376 (local address 127.24.0.190:43253)
0812 01:19:35.087365 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:35.087375 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:35.087411 (+ 36us) server_negotiation.cc:407] Connection header received
0812 01:19:35.087967 (+ 556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:35.087976 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:35.088039 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:35.088222 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:35.088901 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:35.090285 (+ 1384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:35.091040 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:35.091711 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:35.091843 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:35.094669 (+ 2826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:35.094674 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:35.094674 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:35.094688 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:35.096821 (+ 2133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:35.097255 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:35.097258 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:35.097258 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:35.097362 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:35.097597 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:35.097598 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:35.097598 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:35.097887 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:35.098021 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:35.098163 (+ 142us) server_negotiation.cc:299] Negotiation successful
0812 01:19:35.098292 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":88,"threads_started":1}
I20260812 01:19:35.105443 5502 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:19:35.105947 5502 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:19:35.113438 5502 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4e335b200cc4457d9e4f977a36acb97e: Generated new TSK 0
I20260812 01:19:35.113651 5502 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.0.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.0.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.0.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.129.keytab.
Entry for principal kudu/127.24.0.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.129.keytab.
I20260812 01:19:35.143669 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.0.129:0
--local_ip_for_outbound_sockets=127.24.0.129
--webserver_interface=127.24.0.129
--webserver_port=0
--tserver_master_addrs=127.24.0.190:43253
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.129.keytab
--principal=kudu/127.24.0.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:39433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:19:35.228889 5534 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:19:35.229103 5534 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:19:35.229128 5534 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:19:35.230257 5534 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:19:35.230291 5534 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:19:35.230319 5534 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.0.129
I20260812 01:19:35.231715 5534 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:39433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.129.keytab
--principal=kudu/127.24.0.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.0.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.0.129
--webserver_port=0
--tserver_master_addrs=127.24.0.190:43253
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5534
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.0.129
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:19:35.231907 5534 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:19:35.232141 5534 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:19:35.237371 5541 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:35.237401 5543 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:19:35.237531 5534 server_base.cc:1034] running on GCE node
W20260812 01:19:35.237399 5539 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:19:35.237892 5534 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:19:35.238190 5534 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:19:35.239353 5534 hybrid_clock.cc:648] HybridClock initialized: now 1786497575239347 us; error 36 us; skew 500 ppm
Aug 12 01:19:35 dist-test-slave-t3q3 krb5kdc[5353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497575, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:19:35.242056 5534 init.cc:377] Logged in from keytab as kudu/127.24.0.129@KRBTEST.COM (short username kudu)
I20260812 01:19:35.242504 5534 webserver.cc:460] Webserver started at http://127.24.0.129:36413/ using document root <none> and password file <none>
I20260812 01:19:35.242781 5534 fs_manager.cc:359] Metadata directory not provided
I20260812 01:19:35.242853 5534 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:19:35.243002 5534 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:19:35.244153 5534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/ts-0/data/instance:
uuid: "26b22a100506466b834372544a4f43c4"
format_stamp: "Formatted at 2026-08-12 01:19:35 on dist-test-slave-t3q3"
server_key: "f697719a2bcde53369bc98f7c71b36d1"
server_key_iv: "6cfd08b34fd833ae94920cb9fe738dc1"
server_key_version: "encryptionkey@0"
I20260812 01:19:35.244598 5534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/ts-0/wal/instance:
uuid: "26b22a100506466b834372544a4f43c4"
format_stamp: "Formatted at 2026-08-12 01:19:35 on dist-test-slave-t3q3"
server_key: "f697719a2bcde53369bc98f7c71b36d1"
server_key_iv: "6cfd08b34fd833ae94920cb9fe738dc1"
server_key_version: "encryptionkey@0"
I20260812 01:19:35.246016 5534 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 01:19:35.246860 5554 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:35.247040 5534 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 01:19:35.247148 5534 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/ts-0/wal
uuid: "26b22a100506466b834372544a4f43c4"
format_stamp: "Formatted at 2026-08-12 01:19:35 on dist-test-slave-t3q3"
server_key: "f697719a2bcde53369bc98f7c71b36d1"
server_key_iv: "6cfd08b34fd833ae94920cb9fe738dc1"
server_key_version: "encryptionkey@0"
I20260812 01:19:35.247224 5534 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:19:35.262391 5534 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:19:35.264581 5534 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:19:35.264755 5534 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:19:35.265009 5534 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:19:35.265355 5534 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:19:35.265413 5534 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:35.265451 5534 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:19:35.265478 5534 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:35.271692 5534 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.129:39997
I20260812 01:19:35.271755 5743 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.129:39997 every 8 connection(s)
I20260812 01:19:35.272157 5534 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/ts-0/data/info.pb
Aug 12 01:19:35 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497575, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.129@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
I20260812 01:19:35.279613 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 5534
I20260812 01:19:35.279729 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/ts-0/wal/instance
I20260812 01:19:35.279968 24578 external_mini_cluster.cc:1428] Setting key dcbd5bb001e7cf194396b2dded311cfb
I20260812 01:19:35.282162 5493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:35.272950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.129:56425 (local address 127.24.0.190:43253)
0812 01:19:35.273004 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:35.273006 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:35.273246 (+ 240us) server_negotiation.cc:407] Connection header received
0812 01:19:35.273797 (+ 551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:35.273798 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:35.273811 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:35.273864 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:35.274864 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:35.275406 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:35.276509 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:35.276741 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:35.276833 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:35.279034 (+ 2201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:35.279035 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:35.279035 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:35.279044 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:35.280668 (+ 1624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:35.281098 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:35.281099 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:35.281100 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:35.281159 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:35.281466 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:35.281467 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:35.281468 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:35.281595 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:35.281676 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:35.281963 (+ 287us) server_negotiation.cc:299] Negotiation successful
0812 01:19:35.282060 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 01:19:35.282388 5747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:35.272912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:43253 (local address 127.24.0.129:56425)
0812 01:19:35.273117 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:35.273154 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:35.273678 (+ 524us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:35.273943 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:35.273946 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:35.274529 (+ 583us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:35.274754 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:35.274764 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:35.275514 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:35.275515 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:35.276415 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:35.276418 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:35.276847 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:35.276848 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:35.276976 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:35.277463 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:35.277472 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:35.278928 (+ 1456us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:19:35.280765 (+ 1837us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:35.280773 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:35.280775 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:35.280998 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:35.281265 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:35.281266 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:35.281267 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:35.281395 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:35.281668 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:19:35.281676 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:19:35.281818 (+ 142us) client_negotiation.cc:769] Sending connection context
0812 01:19:35.281926 (+ 108us) client_negotiation.cc:240] Negotiation successful
0812 01:19:35.282094 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":103,"threads_started":1}
I20260812 01:19:35.282912 5744 heartbeater.cc:347] Connected to a master server at 127.24.0.190:43253
I20260812 01:19:35.283028 5744 heartbeater.cc:464] Registering TS with master...
I20260812 01:19:35.283246 5744 heartbeater.cc:511] Master 127.24.0.190:43253 requested a full tablet report, sending...
I20260812 01:19:35.283775 5426 ts_manager.cc:194] Registered new tserver with Master: 26b22a100506466b834372544a4f43c4 (127.24.0.129:39997)
I20260812 01:19:35.284484 5426 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.0.129@KRBTEST.COM'} at 127.24.0.129:56425
WARNING: no policy specified for kudu/127.24.0.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.0.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.0.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.130.keytab.
Entry for principal kudu/127.24.0.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.130.keytab.
I20260812 01:19:35.308135 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.0.130:0
--local_ip_for_outbound_sockets=127.24.0.130
--webserver_interface=127.24.0.130
--webserver_port=0
--tserver_master_addrs=127.24.0.190:43253
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.130.keytab
--principal=kudu/127.24.0.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:39433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:19:35.391590 5752 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:19:35.391821 5752 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:19:35.391845 5752 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:19:35.392972 5752 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:19:35.393035 5752 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:19:35.393062 5752 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.0.130
I20260812 01:19:35.394634 5752 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:39433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.130.keytab
--principal=kudu/127.24.0.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.0.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.0.130
--webserver_port=0
--tserver_master_addrs=127.24.0.190:43253
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5752
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.0.130
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:19:35.394855 5752 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:19:35.395092 5752 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 01:19:35.400249 5752 server_base.cc:1034] running on GCE node
W20260812 01:19:35.400202 5758 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:35.400233 5762 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:35.400205 5760 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:19:35.400700 5752 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:19:35.400913 5752 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:19:35.402055 5752 hybrid_clock.cc:648] HybridClock initialized: now 1786497575402040 us; error 28 us; skew 500 ppm
Aug 12 01:19:35 dist-test-slave-t3q3 krb5kdc[5353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497575, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:19:35.408119 5752 init.cc:377] Logged in from keytab as kudu/127.24.0.130@KRBTEST.COM (short username kudu)
I20260812 01:19:35.408545 5752 webserver.cc:460] Webserver started at http://127.24.0.130:44299/ using document root <none> and password file <none>
I20260812 01:19:35.408778 5752 fs_manager.cc:359] Metadata directory not provided
I20260812 01:19:35.408847 5752 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:19:35.408998 5752 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:19:35.410043 5752 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/ts-1/data/instance:
uuid: "93ae86a090914607bc43e96e380152a7"
format_stamp: "Formatted at 2026-08-12 01:19:35 on dist-test-slave-t3q3"
server_key: "90e809d561ec809136d8543133d4a459"
server_key_iv: "a4e73d9166e74124fc37f5c7f1d876ce"
server_key_version: "encryptionkey@0"
I20260812 01:19:35.410411 5752 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/ts-1/wal/instance:
uuid: "93ae86a090914607bc43e96e380152a7"
format_stamp: "Formatted at 2026-08-12 01:19:35 on dist-test-slave-t3q3"
server_key: "90e809d561ec809136d8543133d4a459"
server_key_iv: "a4e73d9166e74124fc37f5c7f1d876ce"
server_key_version: "encryptionkey@0"
I20260812 01:19:35.411787 5752 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 01:19:35.412611 5773 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:35.412789 5752 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:35.412890 5752 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/ts-1/wal
uuid: "93ae86a090914607bc43e96e380152a7"
format_stamp: "Formatted at 2026-08-12 01:19:35 on dist-test-slave-t3q3"
server_key: "90e809d561ec809136d8543133d4a459"
server_key_iv: "a4e73d9166e74124fc37f5c7f1d876ce"
server_key_version: "encryptionkey@0"
I20260812 01:19:35.412976 5752 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:19:35.435465 5752 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:19:35.437366 5752 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:19:35.437479 5752 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:19:35.437678 5752 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:19:35.437942 5752 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:19:35.437973 5752 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:35.437995 5752 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:19:35.438007 5752 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:35.443879 5752 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.130:38135
I20260812 01:19:35.443934 5947 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.130:38135 every 8 connection(s)
I20260812 01:19:35.444262 5752 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/ts-1/data/info.pb
Aug 12 01:19:35 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497575, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.130@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
I20260812 01:19:35.453076 5493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:35.444974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.130:43303 (local address 127.24.0.190:43253)
0812 01:19:35.445035 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:35.445037 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:35.445245 (+ 208us) server_negotiation.cc:407] Connection header received
0812 01:19:35.445720 (+ 475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:35.445721 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:35.445731 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:35.445795 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:35.446740 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:35.447258 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:35.448243 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:35.448434 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:35.448493 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:35.450628 (+ 2135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:35.450629 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:35.450629 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:35.450636 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:35.451907 (+ 1271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:35.452327 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:35.452328 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:35.452329 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:35.452381 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:35.452580 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:35.452580 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:35.452580 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:35.452673 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:35.452719 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:35.452940 (+ 221us) server_negotiation.cc:299] Negotiation successful
0812 01:19:35.453010 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 01:19:35.453259 5953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:35.444937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:43253 (local address 127.24.0.130:43303)
0812 01:19:35.445112 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:35.445153 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:35.445632 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:35.445850 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:35.445853 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:35.446428 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:35.446618 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:35.446622 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:35.447345 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:35.447346 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:35.448156 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:35.448158 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:35.448515 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:35.448516 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:35.448617 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:35.449149 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:35.449157 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:35.450547 (+ 1390us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:19:35.451980 (+ 1433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:35.451986 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:35.451987 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:35.452263 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:35.452441 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:35.452442 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:35.452442 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:35.452539 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:35.452723 (+ 184us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:19:35.452727 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:19:35.452836 (+ 109us) client_negotiation.cc:769] Sending connection context
0812 01:19:35.452916 (+ 80us) client_negotiation.cc:240] Negotiation successful
0812 01:19:35.453044 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":95,"threads_started":1}
I20260812 01:19:35.453783 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 5752
I20260812 01:19:35.453819 5950 heartbeater.cc:347] Connected to a master server at 127.24.0.190:43253
I20260812 01:19:35.453914 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/ts-1/wal/instance
I20260812 01:19:35.453919 5950 heartbeater.cc:464] Registering TS with master...
I20260812 01:19:35.454123 5950 heartbeater.cc:511] Master 127.24.0.190:43253 requested a full tablet report, sending...
I20260812 01:19:35.454149 24578 external_mini_cluster.cc:1428] Setting key bac223ff4bc6aabb1cf27e1b19fe8e73
I20260812 01:19:35.454612 5425 ts_manager.cc:194] Registered new tserver with Master: 93ae86a090914607bc43e96e380152a7 (127.24.0.130:38135)
I20260812 01:19:35.455047 5425 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.0.130@KRBTEST.COM'} at 127.24.0.130:43303
WARNING: no policy specified for kudu/127.24.0.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.0.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.0.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.131.keytab.
Entry for principal kudu/127.24.0.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.131.keytab.
I20260812 01:19:35.482575 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.0.131:0
--local_ip_for_outbound_sockets=127.24.0.131
--webserver_interface=127.24.0.131
--webserver_port=0
--tserver_master_addrs=127.24.0.190:43253
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.131.keytab
--principal=kudu/127.24.0.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:39433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:19:35.566546 5956 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:19:35.566771 5956 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:19:35.566795 5956 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:19:35.568020 5956 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:19:35.568111 5956 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:19:35.568151 5956 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.0.131
I20260812 01:19:35.569639 5956 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:39433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.131.keytab
--principal=kudu/127.24.0.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.0.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.0.131
--webserver_port=0
--tserver_master_addrs=127.24.0.190:43253
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5956
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.0.131
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:19:35.569832 5956 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:19:35.570004 5956 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:19:35.574862 5966 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:35.574898 5967 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:35.574874 5969 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:19:35.574957 5956 server_base.cc:1034] running on GCE node
I20260812 01:19:35.575304 5956 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:19:35.575603 5956 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:19:35.576781 5956 hybrid_clock.cc:648] HybridClock initialized: now 1786497575576761 us; error 46 us; skew 500 ppm
Aug 12 01:19:35 dist-test-slave-t3q3 krb5kdc[5353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497575, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:19:35.579447 5956 init.cc:377] Logged in from keytab as kudu/127.24.0.131@KRBTEST.COM (short username kudu)
I20260812 01:19:35.579880 5956 webserver.cc:460] Webserver started at http://127.24.0.131:44027/ using document root <none> and password file <none>
I20260812 01:19:35.580206 5956 fs_manager.cc:359] Metadata directory not provided
I20260812 01:19:35.580302 5956 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:19:35.580456 5956 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:19:35.581562 5956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/ts-2/data/instance:
uuid: "2daa5e48ddde42e1965ab6163e38ef3d"
format_stamp: "Formatted at 2026-08-12 01:19:35 on dist-test-slave-t3q3"
server_key: "031f2b6b0a0db1d8c21c18d6494a7f73"
server_key_iv: "2e0cbd7a5efe20b803262f0da9d9dd28"
server_key_version: "encryptionkey@0"
I20260812 01:19:35.581944 5956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/ts-2/wal/instance:
uuid: "2daa5e48ddde42e1965ab6163e38ef3d"
format_stamp: "Formatted at 2026-08-12 01:19:35 on dist-test-slave-t3q3"
server_key: "031f2b6b0a0db1d8c21c18d6494a7f73"
server_key_iv: "2e0cbd7a5efe20b803262f0da9d9dd28"
server_key_version: "encryptionkey@0"
I20260812 01:19:35.583412 5956 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 01:19:35.584270 5982 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:35.584468 5956 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 01:19:35.584585 5956 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/ts-2/wal
uuid: "2daa5e48ddde42e1965ab6163e38ef3d"
format_stamp: "Formatted at 2026-08-12 01:19:35 on dist-test-slave-t3q3"
server_key: "031f2b6b0a0db1d8c21c18d6494a7f73"
server_key_iv: "2e0cbd7a5efe20b803262f0da9d9dd28"
server_key_version: "encryptionkey@0"
I20260812 01:19:35.584662 5956 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:19:35.600396 5956 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:19:35.602321 5956 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:19:35.602496 5956 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:19:35.602736 5956 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:19:35.603078 5956 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:19:35.603137 5956 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:35.603179 5956 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:19:35.603207 5956 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:35.609112 5956 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.131:34159
I20260812 01:19:35.609162 6170 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.131:34159 every 8 connection(s)
I20260812 01:19:35.609484 5956 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/ts-2/data/info.pb
Aug 12 01:19:35 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497575, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.131@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
I20260812 01:19:35.618062 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 5956
I20260812 01:19:35.618177 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/ts-2/wal/instance
I20260812 01:19:35.618387 24578 external_mini_cluster.cc:1428] Setting key 2935014120279bf2e83632fc63605559
I20260812 01:19:35.619208 5493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:35.610231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.131:51673 (local address 127.24.0.190:43253)
0812 01:19:35.610293 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:35.610295 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:35.610461 (+ 166us) server_negotiation.cc:407] Connection header received
0812 01:19:35.610939 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:35.610940 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:35.610951 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:35.611017 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:35.611995 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:35.612581 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:35.613757 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:35.613965 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:35.614025 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:35.616343 (+ 2318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:35.616345 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:35.616345 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:35.616352 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:35.617746 (+ 1394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:35.618144 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:35.618145 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:35.618146 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:35.618217 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:35.618491 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:35.618493 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:35.618493 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:35.618666 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:35.618738 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:35.619021 (+ 283us) server_negotiation.cc:299] Negotiation successful
0812 01:19:35.619104 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 01:19:35.619410 6175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:35.610163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:43253 (local address 127.24.0.131:51673)
0812 01:19:35.610330 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:35.610372 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:35.610845 (+ 473us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:35.611122 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:35.611126 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:35.611703 (+ 577us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:35.611893 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:35.611898 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:35.612688 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:35.612689 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:35.613680 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:35.613683 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:35.614052 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:35.614053 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:35.614154 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:35.614706 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:35.614715 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:35.616257 (+ 1542us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:19:35.617820 (+ 1563us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:35.617826 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:35.617828 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:35.618064 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:35.618296 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:35.618296 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:35.618297 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:35.618412 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:35.618739 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:19:35.618746 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:19:35.618887 (+ 141us) client_negotiation.cc:769] Sending connection context
0812 01:19:35.618998 (+ 111us) client_negotiation.cc:240] Negotiation successful
0812 01:19:35.619157 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":95,"threads_started":1}
I20260812 01:19:35.619843 6172 heartbeater.cc:347] Connected to a master server at 127.24.0.190:43253
I20260812 01:19:35.619926 6172 heartbeater.cc:464] Registering TS with master...
I20260812 01:19:35.620144 6172 heartbeater.cc:511] Master 127.24.0.190:43253 requested a full tablet report, sending...
I20260812 01:19:35.620515 5425 ts_manager.cc:194] Registered new tserver with Master: 2daa5e48ddde42e1965ab6163e38ef3d (127.24.0.131:34159)
I20260812 01:19:35.620905 5425 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.0.131@KRBTEST.COM'} at 127.24.0.131:51673
I20260812 01:19:35.630314 24578 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.24.0.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.0.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.0.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/krb5kdc/oryx_127.24.0.129.keytab.
Entry for principal oryx/127.24.0.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/krb5kdc/oryx_127.24.0.129.keytab.
WARNING: no policy specified for oryx/127.24.0.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.0.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.0.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/krb5kdc/oryx_127.24.0.130.keytab.
Entry for principal oryx/127.24.0.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/krb5kdc/oryx_127.24.0.130.keytab.
WARNING: no policy specified for oryx/127.24.0.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.0.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.0.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/krb5kdc/oryx_127.24.0.131.keytab.
Entry for principal oryx/127.24.0.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/krb5kdc/oryx_127.24.0.131.keytab.
I20260812 01:19:35.708802 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 5534
I20260812 01:19:35.728854 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 5752
I20260812 01:19:35.743995 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 5956
I20260812 01:19:35.756418 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 5378
I20260812 01:19:35.767709 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.190:43253
--webserver_interface=127.24.0.190
--webserver_port=37749
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab
--principal=kudu/127.24.0.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:39433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.0.190:43253
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:19:35.853116 6187 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:19:35.853322 6187 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:19:35.853343 6187 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:19:35.854473 6187 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:19:35.854516 6187 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:19:35.854530 6187 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:19:35.854568 6187 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:19:35.854598 6187 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:19:35.856017 6187 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:39433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.0.190:43253
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab
--principal=kudu/127.24.0.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.190:43253
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.0.190
--webserver_port=37749
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6187
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:19:35.856325 6187 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:19:35.856555 6187 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:19:35.861306 6199 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:35.861322 6195 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:35.861322 6197 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:19:35.861400 6187 server_base.cc:1034] running on GCE node
I20260812 01:19:35.861744 6187 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:19:35.862042 6187 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:19:35.863235 6187 hybrid_clock.cc:648] HybridClock initialized: now 1786497575863220 us; error 35 us; skew 500 ppm
Aug 12 01:19:35 dist-test-slave-t3q3 krb5kdc[5353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497575, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:19:35.865759 6187 init.cc:377] Logged in from keytab as kudu/127.24.0.190@KRBTEST.COM (short username kudu)
I20260812 01:19:35.866215 6187 webserver.cc:460] Webserver started at http://127.24.0.190:37749/ using document root <none> and password file <none>
I20260812 01:19:35.866395 6187 fs_manager.cc:359] Metadata directory not provided
I20260812 01:19:35.866448 6187 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:19:35.867787 6187 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:19:35.868604 6212 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:35.868811 6187 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 01:19:35.868904 6187 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/master-0/wal
uuid: "4e335b200cc4457d9e4f977a36acb97e"
format_stamp: "Formatted at 2026-08-12 01:19:35 on dist-test-slave-t3q3"
server_key: "a7c8ca49d649b269cede4633e5a91e5c"
server_key_iv: "c2e8f993c572066cd475b0142569251b"
server_key_version: "encryptionkey@0"
I20260812 01:19:35.869212 6187 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:19:35.879217 6187 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:19:35.881054 6187 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:19:35.881160 6187 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:19:35.885198 6187 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.190:43253
I20260812 01:19:35.885259 6300 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.190:43253 every 8 connection(s)
I20260812 01:19:35.885614 6187 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
I20260812 01:19:35.887045 6301 sys_catalog.cc:263] Verifying existing consensus state
I20260812 01:19:35.887708 6301 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e335b200cc4457d9e4f977a36acb97e: Bootstrap starting.
I20260812 01:19:35.892704 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 6187
I20260812 01:19:35.893070 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.0.129:39997
--local_ip_for_outbound_sockets=127.24.0.129
--tserver_master_addrs=127.24.0.190:43253
--webserver_port=36413
--webserver_interface=127.24.0.129
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.129.keytab
--principal=kudu/127.24.0.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:39433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.24.0.129
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/krb5kdc/oryx_127.24.0.129.keytab with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 01:19:35.896720 6301 log.cc:824] T 00000000000000000000000000000000 P 4e335b200cc4457d9e4f977a36acb97e: Log is configured to *not* fsync() on all Append() calls
I20260812 01:19:35.898432 6301 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e335b200cc4457d9e4f977a36acb97e: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 01:19:35.898741 6301 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e335b200cc4457d9e4f977a36acb97e: Bootstrap complete.
I20260812 01:19:35.901077 6301 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4e335b200cc4457d9e4f977a36acb97e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e335b200cc4457d9e4f977a36acb97e" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 43253 } }
I20260812 01:19:35.901329 6301 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4e335b200cc4457d9e4f977a36acb97e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e335b200cc4457d9e4f977a36acb97e, State: Initialized, Role: FOLLOWER
I20260812 01:19:35.901518 6301 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4e335b200cc4457d9e4f977a36acb97e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e335b200cc4457d9e4f977a36acb97e" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 43253 } }
I20260812 01:19:35.901597 6301 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4e335b200cc4457d9e4f977a36acb97e [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:19:35.901650 6301 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4e335b200cc4457d9e4f977a36acb97e [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:19:35.901705 6301 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4e335b200cc4457d9e4f977a36acb97e [term 1 FOLLOWER]: Advancing to term 2
I20260812 01:19:35.902684 6301 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4e335b200cc4457d9e4f977a36acb97e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e335b200cc4457d9e4f977a36acb97e" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 43253 } }
I20260812 01:19:35.902807 6301 leader_election.cc:302] T 00000000000000000000000000000000 P 4e335b200cc4457d9e4f977a36acb97e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4e335b200cc4457d9e4f977a36acb97e; no voters:
I20260812 01:19:35.903024 6301 leader_election.cc:288] T 00000000000000000000000000000000 P 4e335b200cc4457d9e4f977a36acb97e [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 01:19:35.903061 6307 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4e335b200cc4457d9e4f977a36acb97e [term 2 FOLLOWER]: Leader election won for term 2
I20260812 01:19:35.903193 6307 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4e335b200cc4457d9e4f977a36acb97e [term 2 LEADER]: Becoming Leader. State: Replica: 4e335b200cc4457d9e4f977a36acb97e, State: Running, Role: LEADER
I20260812 01:19:35.903312 6301 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4e335b200cc4457d9e4f977a36acb97e [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:19:35.903350 6307 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4e335b200cc4457d9e4f977a36acb97e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e335b200cc4457d9e4f977a36acb97e" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 43253 } }
I20260812 01:19:35.903716 6308 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e335b200cc4457d9e4f977a36acb97e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "4e335b200cc4457d9e4f977a36acb97e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e335b200cc4457d9e4f977a36acb97e" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 43253 } } }
I20260812 01:19:35.903803 6308 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e335b200cc4457d9e4f977a36acb97e [sys.catalog]: This master's current role is: LEADER
I20260812 01:19:35.904136 6310 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e335b200cc4457d9e4f977a36acb97e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4e335b200cc4457d9e4f977a36acb97e. Latest consensus state: current_term: 2 leader_uuid: "4e335b200cc4457d9e4f977a36acb97e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e335b200cc4457d9e4f977a36acb97e" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 43253 } } }
I20260812 01:19:35.904196 6310 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e335b200cc4457d9e4f977a36acb97e [sys.catalog]: This master's current role is: LEADER
I20260812 01:19:35.904273 6316 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:19:35.904877 6316 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:19:35.905021 6316 catalog_manager.cc:1259] Loaded cluster ID: ede4c36ad3b145609654e79cb5dd8dae
I20260812 01:19:35.905057 6316 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 01:19:35.905957 6316 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:19:35.906157 6316 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 4e335b200cc4457d9e4f977a36acb97e: Loaded TSK: 0
I20260812 01:19:35.906380 6316 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 01:19:35.991564 6303 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:19:35.991798 6303 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:19:35.991840 6303 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:19:35.993078 6303 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:19:35.993144 6303 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:19:35.993198 6303 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.0.129
I20260812 01:19:35.994684 6303 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:39433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/krb5kdc/oryx_127.24.0.129.keytab
--principal=oryx/127.24.0.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.0.129:39997
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.0.129
--webserver_port=36413
--tserver_master_addrs=127.24.0.190:43253
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6303
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.0.129
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:19:35.994962 6303 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:19:35.995203 6303 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 01:19:36.001492 6303 server_base.cc:1034] running on GCE node
W20260812 01:19:36.001389 6341 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:36.001389 6337 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:36.001417 6335 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:19:36.001859 6303 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:19:36.002152 6303 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:19:36.003306 6303 hybrid_clock.cc:648] HybridClock initialized: now 1786497576003291 us; error 33 us; skew 500 ppm
Aug 12 01:19:36 dist-test-slave-t3q3 krb5kdc[5353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497576, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.0.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:19:36.006234 6303 init.cc:377] Logged in from keytab as oryx/127.24.0.129@KRBTEST.COM (short username oryx)
I20260812 01:19:36.006726 6303 webserver.cc:460] Webserver started at http://127.24.0.129:36413/ using document root <none> and password file <none>
I20260812 01:19:36.006992 6303 fs_manager.cc:359] Metadata directory not provided
I20260812 01:19:36.007052 6303 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:19:36.008541 6303 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260812 01:19:36.009209 6358 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:36.009423 6303 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:19:36.009524 6303 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/ts-0/wal
uuid: "26b22a100506466b834372544a4f43c4"
format_stamp: "Formatted at 2026-08-12 01:19:35 on dist-test-slave-t3q3"
server_key: "f697719a2bcde53369bc98f7c71b36d1"
server_key_iv: "6cfd08b34fd833ae94920cb9fe738dc1"
server_key_version: "encryptionkey@0"
I20260812 01:19:36.009891 6303 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:19:36.034579 6303 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:19:36.037228 6303 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:19:36.037397 6303 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:19:36.037669 6303 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:19:36.037997 6303 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:19:36.038053 6303 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:36.038098 6303 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:19:36.038127 6303 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:36.045163 6303 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.129:39997
I20260812 01:19:36.045225 6558 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.129:39997 every 8 connection(s)
I20260812 01:19:36.045614 6303 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/ts-0/data/info.pb
I20260812 01:19:36.049978 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 6303
I20260812 01:19:36.050344 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.0.130:38135
--local_ip_for_outbound_sockets=127.24.0.130
--tserver_master_addrs=127.24.0.190:43253
--webserver_port=44299
--webserver_interface=127.24.0.130
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.130.keytab
--principal=kudu/127.24.0.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:39433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.24.0.130
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/krb5kdc/oryx_127.24.0.130.keytab with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 01:19:36 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.129@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:19:36 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.129@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 01:19:36.054662 6561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:36.046445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:43253 (local address 127.24.0.129:38689)
0812 01:19:36.046660 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:36.046712 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:36.047189 (+ 477us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:36.047591 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:36.047595 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:36.048284 (+ 689us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:36.048486 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:36.048491 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:36.049554 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:36.049555 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:36.050791 (+ 1236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:36.050795 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:36.051367 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:36.051368 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:36.051512 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:36.052167 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:36.052177 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:36.054517 (+ 2340us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.0.190:43253: Server oryx/127.24.0.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":79,"threads_started":1}
W20260812 01:19:36.054932 6562 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:19:36.046487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.129:38689 (local address 127.24.0.190:43253)
0812 01:19:36.046723 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:36.046731 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:36.046787 (+ 56us) server_negotiation.cc:407] Connection header received
0812 01:19:36.047358 (+ 571us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:36.047364 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:36.047407 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:36.047516 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:36.048589 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:36.049462 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:36.050876 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:36.051273 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:36.051354 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:36.054712 (+ 3358us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.0.129:38689: BlockingRecv error: recv got EOF from 127.24.0.129:38689 (error 108)
Metrics: {"server-negotiator.queue_time_us":179,"thread_start_us":117,"threads_started":1}
W20260812 01:19:36.055168 6559 heartbeater.cc:644] Failed to heartbeat to 127.24.0.190:43253 (0 consecutive failures): Not authorized: Failed to ping master at 127.24.0.190:43253: Client connection negotiation failed: client connection to 127.24.0.190:43253: Server oryx/127.24.0.190@KRBTEST.COM not found in Kerberos database
Aug 12 01:19:36 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.129@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:19:36 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.129@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 01:19:36.060186 6561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:36.055401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:43253 (local address 127.24.0.129:43729)
0812 01:19:36.055515 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:36.055524 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:36.055568 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:36.055709 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:36.055710 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:36.055820 (+ 110us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:36.055866 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:36.055867 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:36.056841 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:36.056841 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:36.057316 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:36.057318 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:36.057856 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:36.057857 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:36.057923 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:36.058361 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:36.058365 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:36.060103 (+ 1738us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.0.190:43253: Server oryx/127.24.0.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":99}
W20260812 01:19:36.060256 6562 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:19:36.055469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.129:43729 (local address 127.24.0.190:43253)
0812 01:19:36.055530 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:36.055533 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:36.055559 (+ 26us) server_negotiation.cc:407] Connection header received
0812 01:19:36.055612 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:36.055613 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:36.055627 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:36.055697 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:36.055936 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:36.056758 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:36.057400 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:36.057772 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:36.057851 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:36.060194 (+ 2343us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.0.129:43729: BlockingRecv error: recv got EOF from 127.24.0.129:43729 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Aug 12 01:19:36 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.129@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:19:36 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.129@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 01:19:36.064549 6561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:36.060592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:43253 (local address 127.24.0.129:49293)
0812 01:19:36.060643 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:36.060653 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:36.060714 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:36.060858 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:36.060858 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:36.060984 (+ 126us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:36.061009 (+ 25us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:36.061010 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:36.061604 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:36.061605 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:36.062086 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:36.062088 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:36.062430 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:36.062431 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:36.062485 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:36.062890 (+ 405us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:36.062895 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:36.064473 (+ 1578us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.0.190:43253: Server oryx/127.24.0.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":33}
W20260812 01:19:36.064597 6562 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:19:36.060676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.129:49293 (local address 127.24.0.190:43253)
0812 01:19:36.060737 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:36.060739 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:36.060759 (+ 20us) server_negotiation.cc:407] Connection header received
0812 01:19:36.060778 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:36.060779 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:36.060791 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:36.060854 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:36.061083 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:36.061525 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:36.062167 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:36.062357 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:36.062432 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:36.064538 (+ 2106us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.0.129:49293: BlockingRecv error: recv got EOF from 127.24.0.129:49293 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260812 01:19:36.064733 6559 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 01:19:36.134521 6565 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:19:36.134712 6565 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:19:36.134735 6565 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:19:36.135888 6565 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:19:36.135931 6565 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:19:36.135957 6565 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.0.130
I20260812 01:19:36.137632 6565 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:39433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/krb5kdc/oryx_127.24.0.130.keytab
--principal=oryx/127.24.0.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.0.130:38135
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.0.130
--webserver_port=44299
--tserver_master_addrs=127.24.0.190:43253
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6565
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.0.130
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:19:36.137872 6565 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:19:36.138108 6565 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:19:36.143008 6574 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:36.143019 6573 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:36.143018 6579 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:19:36.143203 6565 server_base.cc:1034] running on GCE node
I20260812 01:19:36.143476 6565 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:19:36.143735 6565 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:19:36.144894 6565 hybrid_clock.cc:648] HybridClock initialized: now 1786497576144882 us; error 43 us; skew 500 ppm
Aug 12 01:19:36 dist-test-slave-t3q3 krb5kdc[5353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497576, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.0.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:19:36.147460 6565 init.cc:377] Logged in from keytab as oryx/127.24.0.130@KRBTEST.COM (short username oryx)
I20260812 01:19:36.147872 6565 webserver.cc:460] Webserver started at http://127.24.0.130:44299/ using document root <none> and password file <none>
I20260812 01:19:36.148166 6565 fs_manager.cc:359] Metadata directory not provided
I20260812 01:19:36.148262 6565 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:19:36.149605 6565 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:19:36.150157 6592 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:36.150324 6565 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 01:19:36.150422 6565 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/ts-1/wal
uuid: "93ae86a090914607bc43e96e380152a7"
format_stamp: "Formatted at 2026-08-12 01:19:35 on dist-test-slave-t3q3"
server_key: "90e809d561ec809136d8543133d4a459"
server_key_iv: "a4e73d9166e74124fc37f5c7f1d876ce"
server_key_version: "encryptionkey@0"
I20260812 01:19:36.150753 6565 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:19:36.158715 6565 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:19:36.161033 6565 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:19:36.161201 6565 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:19:36.161425 6565 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:19:36.161726 6565 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:19:36.161787 6565 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:36.161825 6565 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:19:36.161854 6565 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:36.167579 6565 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.130:38135
I20260812 01:19:36.167685 6781 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.130:38135 every 8 connection(s)
I20260812 01:19:36.168000 6565 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/ts-1/data/info.pb
Aug 12 01:19:36 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.130@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:19:36 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.130@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
W20260812 01:19:36.175148 6562 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:19:36.168756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.130:40341 (local address 127.24.0.190:43253)
0812 01:19:36.168824 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:36.168826 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:36.169004 (+ 178us) server_negotiation.cc:407] Connection header received
0812 01:19:36.169482 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:36.169483 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:36.169497 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:36.169555 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:36.170428 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:36.170909 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:36.172000 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:36.172226 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:36.172281 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:36.175092 (+ 2811us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.0.130:40341: BlockingRecv error: recv got EOF from 127.24.0.130:40341 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 01:19:36.175210 6784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:36.168716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:43253 (local address 127.24.0.130:40341)
0812 01:19:36.168881 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:36.168922 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:36.169399 (+ 477us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:36.169608 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:36.169612 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:36.170168 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:36.170353 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:36.170358 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:36.170987 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:36.170988 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:36.171921 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:36.171923 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:36.172313 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:36.172314 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:36.172414 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:36.172880 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:36.172888 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:36.175019 (+ 2131us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.0.190:43253: Server oryx/127.24.0.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":98,"threads_started":1}
W20260812 01:19:36.175427 6782 heartbeater.cc:644] Failed to heartbeat to 127.24.0.190:43253 (0 consecutive failures): Not authorized: Failed to ping master at 127.24.0.190:43253: Client connection negotiation failed: client connection to 127.24.0.190:43253: Server oryx/127.24.0.190@KRBTEST.COM not found in Kerberos database
I20260812 01:19:36.176769 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 6565
I20260812 01:19:36.177063 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.0.131:34159
--local_ip_for_outbound_sockets=127.24.0.131
--tserver_master_addrs=127.24.0.190:43253
--webserver_port=44027
--webserver_interface=127.24.0.131
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.131.keytab
--principal=kudu/127.24.0.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:39433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.24.0.131
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/krb5kdc/oryx_127.24.0.131.keytab with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 01:19:36 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.130@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:19:36 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.130@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 01:19:36.180254 6784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:36.175680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:43253 (local address 127.24.0.130:53295)
0812 01:19:36.175739 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:36.175750 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:36.175790 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:36.175931 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:36.175931 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:36.176097 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:36.176152 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:36.176154 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:36.176793 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:36.176794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:36.177438 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:36.177439 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:36.177786 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:36.177787 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:36.177850 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:36.178265 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:36.178270 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:36.180161 (+ 1891us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.0.190:43253: Server oryx/127.24.0.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":43}
W20260812 01:19:36.180265 6562 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:19:36.175744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.130:53295 (local address 127.24.0.190:43253)
0812 01:19:36.175790 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:36.175793 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:36.175802 (+ 9us) server_negotiation.cc:407] Connection header received
0812 01:19:36.175846 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:36.175847 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:36.175857 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:36.175927 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:36.176238 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:36.176713 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:36.177528 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:36.177716 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:36.177789 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:36.180222 (+ 2433us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.0.130:53295: BlockingRecv error: recv got EOF from 127.24.0.130:53295 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
Aug 12 01:19:36 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.130@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:19:36 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.130@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 01:19:36.184801 6784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:36.180591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:43253 (local address 127.24.0.130:39161)
0812 01:19:36.180640 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:36.180651 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:36.180701 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:36.180817 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:36.180819 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:36.180937 (+ 118us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:36.180962 (+ 25us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:36.180963 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:36.181568 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:36.181569 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:36.182048 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:36.182049 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:36.182431 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:36.182432 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:36.182490 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:36.182895 (+ 405us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:36.182900 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:36.184723 (+ 1823us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.0.190:43253: Server oryx/127.24.0.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":33}
W20260812 01:19:36.184868 6562 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:19:36.180645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.130:39161 (local address 127.24.0.190:43253)
0812 01:19:36.180686 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:36.180688 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:36.180696 (+ 8us) server_negotiation.cc:407] Connection header received
0812 01:19:36.180753 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:36.180754 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:36.180763 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:36.180806 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:36.181034 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:36.181487 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:36.182146 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:36.182349 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:36.182421 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:36.184802 (+ 2381us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.0.130:39161: BlockingRecv error: recv got EOF from 127.24.0.130:39161 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260812 01:19:36.185077 6782 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 01:19:36.261157 6785 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:19:36.261354 6785 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:19:36.261377 6785 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:19:36.262549 6785 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:19:36.262598 6785 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:19:36.262625 6785 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.0.131
I20260812 01:19:36.264184 6785 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:39433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/krb5kdc/oryx_127.24.0.131.keytab
--principal=oryx/127.24.0.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.0.131:34159
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.0.131
--webserver_port=44027
--tserver_master_addrs=127.24.0.190:43253
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6785
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.0.131
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:19:36.264441 6785 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:19:36.264709 6785 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:19:36.269908 6794 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:36.269886 6798 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:36.269899 6793 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:19:36.270105 6785 server_base.cc:1034] running on GCE node
I20260812 01:19:36.270378 6785 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:19:36.270584 6785 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:19:36.271724 6785 hybrid_clock.cc:648] HybridClock initialized: now 1786497576271713 us; error 29 us; skew 500 ppm
Aug 12 01:19:36 dist-test-slave-t3q3 krb5kdc[5353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497576, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.0.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:19:36.274549 6785 init.cc:377] Logged in from keytab as oryx/127.24.0.131@KRBTEST.COM (short username oryx)
I20260812 01:19:36.275012 6785 webserver.cc:460] Webserver started at http://127.24.0.131:44027/ using document root <none> and password file <none>
I20260812 01:19:36.275274 6785 fs_manager.cc:359] Metadata directory not provided
I20260812 01:19:36.275359 6785 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:19:36.276825 6785 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.000s
I20260812 01:19:36.277453 6809 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:36.277675 6785 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 01:19:36.277755 6785 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/ts-2/wal
uuid: "2daa5e48ddde42e1965ab6163e38ef3d"
format_stamp: "Formatted at 2026-08-12 01:19:35 on dist-test-slave-t3q3"
server_key: "031f2b6b0a0db1d8c21c18d6494a7f73"
server_key_iv: "2e0cbd7a5efe20b803262f0da9d9dd28"
server_key_version: "encryptionkey@0"
I20260812 01:19:36.278146 6785 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:19:36.285331 6785 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:19:36.287122 6785 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:19:36.287284 6785 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:19:36.287504 6785 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:19:36.287811 6785 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:19:36.287845 6785 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:36.287865 6785 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:19:36.287914 6785 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:36.293928 6785 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.131:34159
I20260812 01:19:36.293972 6987 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.131:34159 every 8 connection(s)
I20260812 01:19:36.294328 6785 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786497534195564-24578-0/minicluster-data/ts-2/data/info.pb
Aug 12 01:19:36 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.131@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:19:36 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.131@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
W20260812 01:19:36.301441 6562 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:19:36.295128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.131:43553 (local address 127.24.0.190:43253)
0812 01:19:36.295176 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:36.295178 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:36.295396 (+ 218us) server_negotiation.cc:407] Connection header received
0812 01:19:36.295865 (+ 469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:36.295866 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:36.295875 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:36.295932 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:36.296851 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:36.297337 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:36.298317 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:36.298495 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:36.298547 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:36.301396 (+ 2849us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.0.131:43553: BlockingRecv error: recv got EOF from 127.24.0.131:43553 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 01:19:36.301448 6992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:36.295095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:43253 (local address 127.24.0.131:43553)
0812 01:19:36.295262 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:36.295304 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:36.295781 (+ 477us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:36.295987 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:36.295990 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:36.296576 (+ 586us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:36.296769 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:36.296773 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:36.297431 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:36.297432 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:36.298239 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:36.298241 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:36.298569 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:36.298570 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:36.298666 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:36.299105 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:36.299113 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:36.301313 (+ 2200us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.0.190:43253: Server oryx/127.24.0.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":97,"threads_started":1}
W20260812 01:19:36.301789 6988 heartbeater.cc:644] Failed to heartbeat to 127.24.0.190:43253 (0 consecutive failures): Not authorized: Failed to ping master at 127.24.0.190:43253: Client connection negotiation failed: client connection to 127.24.0.190:43253: Server oryx/127.24.0.190@KRBTEST.COM not found in Kerberos database
I20260812 01:19:36.303576 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 6785
Aug 12 01:19:36 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.131@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:19:36 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.131@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 01:19:36.306787 6992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:36.302007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:43253 (local address 127.24.0.131:48519)
0812 01:19:36.302062 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:36.302073 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:36.302121 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:36.302371 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:36.302372 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:36.302503 (+ 131us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:36.302545 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:36.302546 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:36.303238 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:36.303239 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:36.304015 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:36.304018 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:36.304449 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:36.304450 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:36.304532 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:36.304992 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:36.304997 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:36.306711 (+ 1714us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.0.190:43253: Server oryx/127.24.0.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":37}
W20260812 01:19:36.306843 6562 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:19:36.302215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.131:48519 (local address 127.24.0.190:43253)
0812 01:19:36.302263 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:36.302264 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:36.302270 (+ 6us) server_negotiation.cc:407] Connection header received
0812 01:19:36.302287 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:36.302288 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:36.302296 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:36.302364 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:36.302644 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:36.303165 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:36.304141 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:36.304323 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:36.304407 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:36.306777 (+ 2370us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.0.131:48519: BlockingRecv error: recv got EOF from 127.24.0.131:48519 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
Aug 12 01:19:36 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.131@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:19:36 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.131@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 01:19:36.311255 6992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:36.307186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:43253 (local address 127.24.0.131:54297)
0812 01:19:36.307246 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:36.307256 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:36.307295 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:36.307404 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:36.307405 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:36.307529 (+ 124us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:36.307554 (+ 25us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:36.307555 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:36.308177 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:36.308178 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:36.308715 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:36.308717 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:36.309201 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:36.309203 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:36.309257 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:36.309612 (+ 355us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:36.309617 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:36.311178 (+ 1561us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.0.190:43253: Server oryx/127.24.0.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":42}
W20260812 01:19:36.311316 6562 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:19:36.307265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.131:54297 (local address 127.24.0.190:43253)
0812 01:19:36.307305 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:36.307307 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:36.307316 (+ 9us) server_negotiation.cc:407] Connection header received
0812 01:19:36.307338 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:36.307339 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:36.307347 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:36.307392 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:36.307633 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:36.308102 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:36.308822 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:36.309113 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:36.309189 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:36.311252 (+ 2063us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.0.131:54297: BlockingRecv error: recv got EOF from 127.24.0.131:54297 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 01:19:36.311570 6996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:36.303973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41384 (local address 127.24.0.190:43253)
0812 01:19:36.304097 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:36.304099 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:36.304237 (+ 138us) server_negotiation.cc:407] Connection header received
0812 01:19:36.304368 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:36.304369 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:36.304381 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:36.304440 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:36.305057 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:36.305629 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:36.306662 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:36.306884 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:36.306992 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:36.308345 (+ 1353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:36.308350 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:36.308351 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:36.308362 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:36.310047 (+ 1685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:36.310449 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:36.310453 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:36.310453 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:36.310556 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:36.310776 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:36.310777 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:36.310778 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:36.311064 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:36.311222 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:36.311348 (+ 126us) server_negotiation.cc:299] Negotiation successful
0812 01:19:36.311463 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":59,"threads_started":1}
W20260812 01:19:36.311650 6988 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 12 01:19:37 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.129@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:19:37 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.129@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 01:19:37.069916 6997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:37.065113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:43253 (local address 127.24.0.129:51009)
0812 01:19:37.065246 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:37.065258 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:37.065334 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:37.065476 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:37.065476 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:37.065646 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:37.065706 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:37.065707 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:37.066345 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:37.066345 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:37.066974 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:37.066978 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:37.067301 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:37.067302 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:37.067341 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:37.067839 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:37.067843 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:37.069788 (+ 1945us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.0.190:43253: Server oryx/127.24.0.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":58,"threads_started":1}
W20260812 01:19:37.069972 7000 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:19:37.065177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.129:51009 (local address 127.24.0.190:43253)
0812 01:19:37.065339 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:37.065340 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:37.065369 (+ 29us) server_negotiation.cc:407] Connection header received
0812 01:19:37.065392 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:37.065393 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:37.065403 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:37.065457 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:37.065775 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:37.066278 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:37.067050 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:37.067238 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:37.067288 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:37.069891 (+ 2603us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.0.129:51009: BlockingRecv error: recv got EOF from 127.24.0.129:51009 (error 108)
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":91,"threads_started":1}
Aug 12 01:19:37 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.130@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:19:37 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.130@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 01:19:37.190337 7001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:37.185495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:43253 (local address 127.24.0.130:60651)
0812 01:19:37.185694 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:37.185706 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:37.185781 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:37.185938 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:37.185940 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:37.186132 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:37.186180 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:37.186182 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:37.186860 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:37.186861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:37.187410 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:37.187412 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:37.187823 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:37.187825 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:37.187886 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:37.188385 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:37.188390 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:37.190228 (+ 1838us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.0.190:43253: Server oryx/127.24.0.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":111,"threads_started":1}
W20260812 01:19:37.190387 7000 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:19:37.185594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.130:60651 (local address 127.24.0.190:43253)
0812 01:19:37.185670 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:37.185672 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:37.185775 (+ 103us) server_negotiation.cc:407] Connection header received
0812 01:19:37.185838 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:37.185839 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:37.185854 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:37.185929 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:37.186266 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:37.186771 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:37.187501 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:37.187705 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:37.187824 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:37.190317 (+ 2493us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.0.130:60651: BlockingRecv error: recv got EOF from 127.24.0.130:60651 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 12 01:19:37 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.131@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:19:37 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.131@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 01:19:37.316597 7003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:37.312041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:43253 (local address 127.24.0.131:56659)
0812 01:19:37.312220 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:37.312231 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:37.312287 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:37.312470 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:37.312471 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:37.312649 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:37.312699 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:37.312700 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:37.313339 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:37.313340 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:37.313910 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:37.313912 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:37.314226 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:37.314227 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:37.314265 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:37.314784 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:37.314789 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:37.316525 (+ 1736us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.0.190:43253: Server oryx/127.24.0.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":110,"threads_started":1}
W20260812 01:19:37.316659 7000 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:19:37.312107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.131:56659 (local address 127.24.0.190:43253)
0812 01:19:37.312174 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:37.312177 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:37.312287 (+ 110us) server_negotiation.cc:407] Connection header received
0812 01:19:37.312330 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:37.312331 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:37.312381 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:37.312441 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:37.312765 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:37.313272 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:37.313982 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:37.314162 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:37.314226 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:37.316612 (+ 2386us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.0.131:56659: BlockingRecv error: recv got EOF from 127.24.0.131:56659 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 12 01:19:38 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.129@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:19:38 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.129@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 01:19:38.075371 7008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:38.070674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:43253 (local address 127.24.0.129:34573)
0812 01:19:38.070826 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:38.070836 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:38.070889 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:38.071009 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:38.071009 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:38.071166 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:38.071218 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:38.071219 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:38.071947 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:38.071948 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:38.072586 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:38.072589 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:38.072997 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:38.072997 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:38.073037 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:38.073446 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:38.073450 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:38.075265 (+ 1815us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.0.190:43253: Server oryx/127.24.0.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":96,"threads_started":1}
W20260812 01:19:38.075414 7011 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:19:38.070768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.129:34573 (local address 127.24.0.190:43253)
0812 01:19:38.070881 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:38.070887 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:38.070896 (+ 9us) server_negotiation.cc:407] Connection header received
0812 01:19:38.070937 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:38.070937 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:38.070948 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:38.071005 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:38.071303 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:38.071858 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:38.072680 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:38.072920 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:38.072978 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:38.075341 (+ 2363us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.0.129:34573: BlockingRecv error: recv got EOF from 127.24.0.129:34573 (error 108)
Metrics: {"server-negotiator.queue_time_us":89,"thread_start_us":58,"threads_started":1}
Aug 12 01:19:38 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.130@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:19:38 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.130@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 01:19:38.195333 7012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:38.190803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:43253 (local address 127.24.0.130:60177)
0812 01:19:38.190940 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:38.190951 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:38.191034 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:38.191173 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:38.191174 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:38.191341 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:38.191392 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:38.191394 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:38.192041 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:38.192042 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:38.192676 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:38.192678 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:38.193038 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:38.193039 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:38.193081 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:38.193510 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:38.193515 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:38.195268 (+ 1753us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.0.190:43253: Server oryx/127.24.0.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":88,"threads_started":1}
W20260812 01:19:38.195472 7011 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:19:38.190982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.130:60177 (local address 127.24.0.190:43253)
0812 01:19:38.191033 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:38.191035 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:38.191043 (+ 8us) server_negotiation.cc:407] Connection header received
0812 01:19:38.191089 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:38.191090 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:38.191101 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:38.191153 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:38.191457 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:38.191945 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:38.192751 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:38.192938 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:38.193003 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:38.195387 (+ 2384us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.0.130:60177: BlockingRecv error: recv got EOF from 127.24.0.130:60177 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Aug 12 01:19:38 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.131@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:19:38 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.131@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 01:19:38.322199 7015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:38.317127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:43253 (local address 127.24.0.131:37897)
0812 01:19:38.317316 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:38.317330 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:38.317405 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:38.317541 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:38.317542 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:38.317718 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:38.317770 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:38.317772 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:38.318660 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:38.318662 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:38.319474 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:38.319476 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:38.319804 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:38.319805 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:38.319844 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:38.320372 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:38.320377 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:38.322132 (+ 1755us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.0.190:43253: Server oryx/127.24.0.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":78,"threads_started":1}
W20260812 01:19:38.322409 7011 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:19:38.317289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.131:37897 (local address 127.24.0.190:43253)
0812 01:19:38.317354 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:38.317356 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:38.317387 (+ 31us) server_negotiation.cc:407] Connection header received
0812 01:19:38.317451 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:38.317452 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:38.317466 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:38.317516 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:38.317832 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:38.318558 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:38.319549 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:38.319736 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:38.319792 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:38.322339 (+ 2547us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.0.131:37897: BlockingRecv error: recv got EOF from 127.24.0.131:37897 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 12 01:19:39 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.129@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:19:39 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.129@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 01:19:39.080556 7016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:39.075968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:43253 (local address 127.24.0.129:55725)
0812 01:19:39.076155 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:39.076167 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:39.076224 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:39.076377 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:39.076378 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:39.076547 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:39.076598 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:39.076600 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:39.077250 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:39.077251 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:39.077783 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:39.077785 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:39.078102 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:39.078103 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:39.078142 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:39.078632 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:39.078637 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:39.080482 (+ 1845us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.0.190:43253: Server oryx/127.24.0.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":130,"threads_started":1}
W20260812 01:19:39.080703 7017 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:19:39.076027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.129:55725 (local address 127.24.0.190:43253)
0812 01:19:39.076201 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:39.076203 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:39.076213 (+ 10us) server_negotiation.cc:407] Connection header received
0812 01:19:39.076284 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:39.076285 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:39.076297 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:39.076370 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:39.076660 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:39.077174 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:39.077851 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:39.078039 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:39.078099 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:39.080605 (+ 2506us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.0.129:55725: BlockingRecv error: recv got EOF from 127.24.0.129:55725 (error 108)
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":123,"threads_started":1}
Aug 12 01:19:39 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.130@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:19:39 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.130@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 01:19:39.200296 7018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:39.195796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:43253 (local address 127.24.0.130:37935)
0812 01:19:39.195983 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:39.195995 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:39.196134 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:39.196283 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:39.196284 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:39.196459 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:39.196510 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:39.196512 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:39.197131 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:39.197132 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:39.197656 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:39.197658 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:39.197975 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:39.197976 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:39.198022 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:39.198469 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:39.198473 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:39.200227 (+ 1754us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.0.190:43253: Server oryx/127.24.0.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":114,"threads_started":1}
W20260812 01:19:39.200392 7017 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:19:39.195864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.130:37935 (local address 127.24.0.190:43253)
0812 01:19:39.195929 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:39.195932 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:39.196078 (+ 146us) server_negotiation.cc:407] Connection header received
0812 01:19:39.196198 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:39.196199 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:39.196211 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:39.196260 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:39.196576 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:39.197061 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:39.197726 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:39.197908 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:39.197958 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:39.200325 (+ 2367us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.0.130:37935: BlockingRecv error: recv got EOF from 127.24.0.130:37935 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Aug 12 01:19:39 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.131@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:19:39 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.131@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 01:19:39.327164 7021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:39.322745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:43253 (local address 127.24.0.131:35277)
0812 01:19:39.322933 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:39.322945 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:39.323014 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:39.323148 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:39.323149 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:39.323321 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:39.323371 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:39.323373 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:39.323985 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:39.323985 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:39.324554 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:39.324556 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:39.324866 (+ 310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:39.324866 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:39.324909 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:39.325362 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:39.325366 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:39.327096 (+ 1730us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.0.190:43253: Server oryx/127.24.0.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":123,"threads_started":1}
W20260812 01:19:39.327276 7017 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:19:39.322789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.131:35277 (local address 127.24.0.190:43253)
0812 01:19:39.322861 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:39.322863 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:39.323017 (+ 154us) server_negotiation.cc:407] Connection header received
0812 01:19:39.323061 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:39.323062 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:39.323076 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:39.323125 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:39.323435 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:39.323921 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:39.324620 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:39.324799 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:39.324852 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:39.327203 (+ 2351us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.0.131:35277: BlockingRecv error: recv got EOF from 127.24.0.131:35277 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 12 01:19:40 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.129@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:19:40 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.129@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 01:19:40.085685 7023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:40.081122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:43253 (local address 127.24.0.129:50913)
0812 01:19:40.081253 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:40.081265 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:40.081325 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:40.081461 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:40.081462 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:40.081622 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:40.081674 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:40.081675 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:40.082321 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:40.082322 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:40.082847 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:40.082849 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:40.083190 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:40.083191 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:40.083229 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:40.083742 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:40.083747 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:40.085621 (+ 1874us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.0.190:43253: Server oryx/127.24.0.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":103,"threads_started":1}
W20260812 01:19:40.085824 7024 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:19:40.081210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.129:50913 (local address 127.24.0.190:43253)
0812 01:19:40.081339 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:40.081341 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:40.081348 (+ 7us) server_negotiation.cc:407] Connection header received
0812 01:19:40.081371 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:40.081371 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:40.081381 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:40.081444 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:40.081751 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:40.082255 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:40.082923 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:40.083123 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:40.083180 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:40.085728 (+ 2548us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.0.129:50913: BlockingRecv error: recv got EOF from 127.24.0.129:50913 (error 108)
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":64,"threads_started":1}
Aug 12 01:19:40 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.130@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:19:40 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.130@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 01:19:40.205791 7028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:40.200841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:43253 (local address 127.24.0.130:48263)
0812 01:19:40.201029 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:40.201041 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:40.201131 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:40.201313 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:40.201314 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:40.201501 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:40.201549 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:40.201551 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:40.202253 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:40.202254 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:40.202786 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:40.202788 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:40.203184 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:40.203185 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:40.203243 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:40.203778 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:40.203782 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:40.205694 (+ 1912us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.0.190:43253: Server oryx/127.24.0.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":116,"threads_started":1}
W20260812 01:19:40.205852 7024 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:19:40.200937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.130:48263 (local address 127.24.0.190:43253)
0812 01:19:40.201014 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:40.201016 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:40.201109 (+ 93us) server_negotiation.cc:407] Connection header received
0812 01:19:40.201217 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:40.201218 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:40.201231 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:40.201294 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:40.201645 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:40.202164 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:40.202880 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:40.203105 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:40.203185 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:40.205786 (+ 2601us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.0.130:48263: BlockingRecv error: recv got EOF from 127.24.0.130:48263 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 12 01:19:40 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.131@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:19:40 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.131@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 01:19:40.332504 7029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:40.327733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:43253 (local address 127.24.0.131:45215)
0812 01:19:40.327877 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:40.327912 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:40.327986 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:40.328197 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:40.328198 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:40.328431 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:40.328495 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:40.328497 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:40.329177 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:40.329178 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:40.329718 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:40.329720 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:40.330082 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:40.330083 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:40.330137 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:40.330628 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:40.330633 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:40.332445 (+ 1812us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.0.190:43253: Server oryx/127.24.0.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":87,"threads_started":1}
W20260812 01:19:40.332647 7024 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:19:40.327866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.131:45215 (local address 127.24.0.190:43253)
0812 01:19:40.327949 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:40.327952 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:40.327967 (+ 15us) server_negotiation.cc:407] Connection header received
0812 01:19:40.328097 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:40.328098 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:40.328114 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:40.328187 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:40.328577 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:40.329089 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:40.329808 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:40.330004 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:40.330107 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:40.332562 (+ 2455us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.0.131:45215: BlockingRecv error: recv got EOF from 127.24.0.131:45215 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 12 01:19:41 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.129@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:19:41 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.129@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 01:19:41.095580 7031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:41.086167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:43253 (local address 127.24.0.129:40667)
0812 01:19:41.086331 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:41.086341 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:41.086419 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:41.086563 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:41.086563 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:41.086739 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:41.086793 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:41.086795 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:41.087523 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:41.087524 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:41.088130 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:41.088132 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:41.088486 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:41.088487 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:41.088538 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:41.093629 (+ 5091us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:41.093634 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:41.095520 (+ 1886us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.0.190:43253: Server oryx/127.24.0.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":97,"threads_started":1}
W20260812 01:19:41.095738 7032 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:19:41.086242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.129:40667 (local address 127.24.0.190:43253)
0812 01:19:41.086357 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:41.086359 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:41.086401 (+ 42us) server_negotiation.cc:407] Connection header received
0812 01:19:41.086471 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:41.086472 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:41.086484 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:41.086545 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:41.086886 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:41.087424 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:41.088213 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:41.088410 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:41.088467 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:41.095637 (+ 7170us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.0.129:40667: BlockingRecv error: recv got EOF from 127.24.0.129:40667 (error 108)
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":59,"threads_started":1}
Aug 12 01:19:41 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.130@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:19:41 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.130@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 01:19:41.211016 7033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:41.206259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:43253 (local address 127.24.0.130:60869)
0812 01:19:41.206399 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:41.206411 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:41.206477 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:41.206640 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:41.206641 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:41.206835 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:41.206886 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:41.206888 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:41.207588 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:41.207590 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:41.208256 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:41.208260 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:41.208677 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:41.208678 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:41.208738 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:41.209188 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:41.209193 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:41.210918 (+ 1725us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.0.190:43253: Server oryx/127.24.0.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":84,"threads_started":1}
W20260812 01:19:41.211064 7032 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:19:41.206399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.130:60869 (local address 127.24.0.190:43253)
0812 01:19:41.206453 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:41.206455 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:41.206466 (+ 11us) server_negotiation.cc:407] Connection header received
0812 01:19:41.206539 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:41.206540 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:41.206554 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:41.206634 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:41.206969 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:41.207481 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:41.208378 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:41.208584 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:41.208676 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:41.211003 (+ 2327us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.0.130:60869: BlockingRecv error: recv got EOF from 127.24.0.130:60869 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
Aug 12 01:19:41 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.131@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:19:41 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.131@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 01:19:41.337896 7034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:41.333170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:43253 (local address 127.24.0.131:59917)
0812 01:19:41.333322 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:41.333332 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:41.333394 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:41.333535 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:41.333536 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:41.333714 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:41.333766 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:41.333768 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:41.334391 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:41.334392 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:41.335011 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:41.335014 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:41.335445 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:41.335446 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:41.335509 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:41.335969 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:41.335974 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:41.337838 (+ 1864us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.0.190:43253: Server oryx/127.24.0.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":91,"threads_started":1}
W20260812 01:19:41.338025 7032 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:19:41.333232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.131:59917 (local address 127.24.0.190:43253)
0812 01:19:41.333305 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:41.333308 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:41.333393 (+ 85us) server_negotiation.cc:407] Connection header received
0812 01:19:41.333455 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:41.333456 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:41.333470 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:41.333514 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:41.333827 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:41.334327 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:41.335135 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:41.335364 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:41.335443 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:41.337926 (+ 2483us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.0.131:59917: BlockingRecv error: recv got EOF from 127.24.0.131:59917 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 12 01:19:42 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.129@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:19:42 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.129@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 01:19:42.101150 7035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:42.096161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:43253 (local address 127.24.0.129:47843)
0812 01:19:42.096337 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:42.096349 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:42.096414 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:42.096568 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:42.096568 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:42.096738 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:42.096793 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:42.096795 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:42.097527 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:42.097528 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:42.098262 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:42.098265 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:42.098617 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:42.098618 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:42.098659 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:42.099234 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:42.099239 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:42.101068 (+ 1829us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.0.190:43253: Server oryx/127.24.0.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":103,"threads_started":1}
W20260812 01:19:42.101368 7036 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:19:42.096225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.129:47843 (local address 127.24.0.190:43253)
0812 01:19:42.096346 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:42.096348 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:42.096399 (+ 51us) server_negotiation.cc:407] Connection header received
0812 01:19:42.096479 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:42.096479 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:42.096490 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:42.096547 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:42.096863 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:42.097383 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:42.098345 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:42.098540 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:42.098596 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:42.101172 (+ 2576us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.0.129:47843: BlockingRecv error: recv got EOF from 127.24.0.129:47843 (error 108)
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":52,"threads_started":1}
Aug 12 01:19:42 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.130@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:19:42 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.130@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 01:19:42.217046 7037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:42.211579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:43253 (local address 127.24.0.130:54379)
0812 01:19:42.211747 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:42.211757 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:42.211817 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:42.211969 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:42.211969 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:42.212208 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:42.212259 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:42.212261 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:42.213283 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:42.213284 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:42.213936 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:42.213939 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:42.214420 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:42.214421 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:42.214464 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:42.214990 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:42.214995 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:42.216986 (+ 1991us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.0.190:43253: Server oryx/127.24.0.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":99,"threads_started":1}
W20260812 01:19:42.217178 7036 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:19:42.211685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.130:54379 (local address 127.24.0.190:43253)
0812 01:19:42.211739 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:42.211741 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:42.211804 (+ 63us) server_negotiation.cc:407] Connection header received
0812 01:19:42.211886 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:42.211887 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:42.211900 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:42.211948 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:42.212347 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:42.213178 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:42.214024 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:42.214339 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:42.214422 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:42.217092 (+ 2670us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.0.130:54379: BlockingRecv error: recv got EOF from 127.24.0.130:54379 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
Aug 12 01:19:42 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.131@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:19:42 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.131@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 01:19:42.343113 7038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:42.338401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:43253 (local address 127.24.0.131:59443)
0812 01:19:42.338585 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:42.338597 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:42.338659 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:42.338774 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:42.338774 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:42.338942 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:42.338993 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:42.338994 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:42.339692 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:42.339693 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:42.340317 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:42.340320 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:42.340714 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:42.340715 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:42.340767 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:42.341259 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:42.341264 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:42.343055 (+ 1791us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.0.190:43253: Server oryx/127.24.0.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":95,"threads_started":1}
W20260812 01:19:42.343329 7036 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:19:42.338480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.131:59443 (local address 127.24.0.190:43253)
0812 01:19:42.338554 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:42.338556 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:42.338643 (+ 87us) server_negotiation.cc:407] Connection header received
0812 01:19:42.338713 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:42.338714 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:42.338726 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:42.338768 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:42.339117 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:42.339618 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:42.340414 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:42.340620 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:42.340686 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:42.343237 (+ 2551us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.0.131:59443: BlockingRecv error: recv got EOF from 127.24.0.131:59443 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 12 01:19:43 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.129@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:19:43 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.129@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 01:19:43.106802 7041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:43.101720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:43253 (local address 127.24.0.129:52279)
0812 01:19:43.101901 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:43.101916 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:43.101989 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:43.102163 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:43.102164 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:43.102339 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:43.102390 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:43.102392 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:43.103057 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:43.103058 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:43.103641 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:43.103643 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:43.104034 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:43.104036 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:43.104164 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:43.104833 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:43.104838 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:43.106704 (+ 1866us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.0.190:43253: Server oryx/127.24.0.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":77,"threads_started":1}
W20260812 01:19:43.106906 7042 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:19:43.101833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.129:52279 (local address 127.24.0.190:43253)
0812 01:19:43.101975 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:43.101977 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:43.101987 (+ 10us) server_negotiation.cc:407] Connection header received
0812 01:19:43.102049 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:43.102049 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:43.102060 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:43.102142 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:43.102453 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:43.102975 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:43.103714 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:43.103914 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:43.103973 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:43.106827 (+ 2854us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.0.129:52279: BlockingRecv error: recv got EOF from 127.24.0.129:52279 (error 108)
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":79,"threads_started":1}
Aug 12 01:19:43 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.130@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:19:43 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.130@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 01:19:43.222394 7044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:43.217529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:43253 (local address 127.24.0.130:57087)
0812 01:19:43.217738 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:43.217750 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:43.217827 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:43.217986 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:43.217988 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:43.218192 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:43.218245 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:43.218247 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:43.218921 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:43.218923 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:43.219492 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:43.219494 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:43.219877 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:43.219879 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:43.219947 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:43.220431 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:43.220436 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:43.222293 (+ 1857us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.0.190:43253: Server oryx/127.24.0.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":98,"threads_started":1}
W20260812 01:19:43.222440 7042 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:19:43.217621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.130:57087 (local address 127.24.0.190:43253)
0812 01:19:43.217729 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:43.217731 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:43.217806 (+ 75us) server_negotiation.cc:407] Connection header received
0812 01:19:43.217887 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:43.217888 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:43.217902 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:43.217981 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:43.218332 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:43.218836 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:43.219597 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:43.219793 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:43.219866 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:43.222376 (+ 2510us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.0.130:57087: BlockingRecv error: recv got EOF from 127.24.0.130:57087 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 12 01:19:43 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.131@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:19:43 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.131@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 01:19:43.348825 7045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:43.343653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:43253 (local address 127.24.0.131:36891)
0812 01:19:43.343808 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:43.343821 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:43.343898 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:43.344084 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:43.344085 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:43.344262 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:43.344312 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:43.344314 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:43.344983 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:43.344984 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:43.345802 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:43.345806 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:43.346211 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:43.346213 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:43.346262 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:43.346816 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:43.346821 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:43.348767 (+ 1946us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.0.190:43253: Server oryx/127.24.0.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":77,"threads_started":1}
W20260812 01:19:43.348970 7042 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:19:43.343751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.131:36891 (local address 127.24.0.190:43253)
0812 01:19:43.343830 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:43.343837 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:43.343885 (+ 48us) server_negotiation.cc:407] Connection header received
0812 01:19:43.343956 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:43.343956 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:43.343969 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:43.344011 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:43.344384 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:43.344910 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:43.345874 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:43.346109 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:43.346165 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:43.348884 (+ 2719us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.0.131:36891: BlockingRecv error: recv got EOF from 127.24.0.131:36891 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 12 01:19:44 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.129@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:19:44 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.129@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 01:19:44.111919 7048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:44.107292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:43253 (local address 127.24.0.129:36513)
0812 01:19:44.107417 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:44.107436 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:44.107486 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:44.107663 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:44.107664 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:44.107836 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:44.107889 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:44.107890 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:44.108582 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:44.108583 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:44.109126 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:44.109128 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:44.109562 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:44.109563 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:44.109633 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:44.110056 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:44.110060 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:44.111858 (+ 1798us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.0.190:43253: Server oryx/127.24.0.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":69,"threads_started":1}
W20260812 01:19:44.112066 7051 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:19:44.107405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.129:36513 (local address 127.24.0.190:43253)
0812 01:19:44.107533 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:44.107535 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:44.107544 (+ 9us) server_negotiation.cc:407] Connection header received
0812 01:19:44.107567 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:44.107568 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:44.107578 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:44.107644 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:44.107953 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:44.108491 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:44.109265 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:44.109479 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:44.109561 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:44.111974 (+ 2413us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.0.129:36513: BlockingRecv error: recv got EOF from 127.24.0.129:36513 (error 108)
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":73,"threads_started":1}
Aug 12 01:19:44 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.130@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:19:44 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.130@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 01:19:44.227872 7053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:44.222864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:43253 (local address 127.24.0.130:40021)
0812 01:19:44.223064 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:44.223078 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:44.223172 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:44.223357 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:44.223358 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:44.223550 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:44.223600 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:44.223602 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:44.224484 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:44.224485 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:44.225013 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:44.225015 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:44.225424 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:44.225425 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:44.225491 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:44.225935 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:44.225940 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:44.227788 (+ 1848us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.0.190:43253: Server oryx/127.24.0.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":105,"threads_started":1}
W20260812 01:19:44.228012 7051 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:19:44.222959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.130:40021 (local address 127.24.0.190:43253)
0812 01:19:44.223046 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:44.223048 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:44.223152 (+ 104us) server_negotiation.cc:407] Connection header received
0812 01:19:44.223251 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:44.223252 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:44.223273 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:44.223350 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:44.223682 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:44.224394 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:44.225134 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:44.225336 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:44.225422 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:44.227945 (+ 2523us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.0.130:40021: BlockingRecv error: recv got EOF from 127.24.0.130:40021 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 12 01:19:44 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.131@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:19:44 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.131@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 01:19:44.354060 7055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:44.349266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:43253 (local address 127.24.0.131:40457)
0812 01:19:44.349461 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:44.349473 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:44.349540 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:44.349677 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:44.349677 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:44.349844 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:44.349895 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:44.349896 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:44.350541 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:44.350541 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:44.351063 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:44.351065 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:44.351390 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:44.351390 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:44.351430 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:44.351934 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:44.351940 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:44.353980 (+ 2040us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.0.190:43253: Server oryx/127.24.0.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":110,"threads_started":1}
W20260812 01:19:44.354182 7051 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:19:44.349361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.131:40457 (local address 127.24.0.190:43253)
0812 01:19:44.349441 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:44.349443 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:44.349524 (+ 81us) server_negotiation.cc:407] Connection header received
0812 01:19:44.349599 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:44.349600 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:44.349614 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:44.349658 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:44.349963 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:44.350461 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:44.351137 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:44.351320 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:44.351375 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:44.354081 (+ 2706us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.0.131:40457: BlockingRecv error: recv got EOF from 127.24.0.131:40457 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 12 01:19:45 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.129@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:19:45 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.129@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 01:19:45.117415 7056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:45.112449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:43253 (local address 127.24.0.129:58123)
0812 01:19:45.112667 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:45.112680 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:45.112746 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:45.112904 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:45.112905 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:45.113086 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:45.113137 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:45.113139 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:45.113910 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:45.113910 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:45.114437 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:45.114438 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:45.114796 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:45.114797 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:45.114836 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:45.115380 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:45.115385 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:45.117356 (+ 1971us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.0.190:43253: Server oryx/127.24.0.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":143,"threads_started":1}
W20260812 01:19:45.117622 7057 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:19:45.112565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.129:58123 (local address 127.24.0.190:43253)
0812 01:19:45.112713 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:45.112715 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:45.112729 (+ 14us) server_negotiation.cc:407] Connection header received
0812 01:19:45.112794 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:45.112795 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:45.112805 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:45.112885 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:45.113206 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:45.113839 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:45.114529 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:45.114727 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:45.114786 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:45.117531 (+ 2745us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.0.129:58123: BlockingRecv error: recv got EOF from 127.24.0.129:58123 (error 108)
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":92,"threads_started":1}
Aug 12 01:19:45 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.130@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:19:45 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.130@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 01:19:45.233239 7058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:45.228416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:43253 (local address 127.24.0.130:41701)
0812 01:19:45.228558 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:45.228577 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:45.228637 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:45.228769 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:45.228769 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:45.228941 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:45.228993 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:45.228994 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:45.229638 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:45.229639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:45.230160 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:45.230162 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:45.230522 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:45.230523 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:45.230575 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:45.231158 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:45.231163 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:45.233179 (+ 2016us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.0.190:43253: Server oryx/127.24.0.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":70,"threads_started":1}
W20260812 01:19:45.233377 7057 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:19:45.228511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.130:41701 (local address 127.24.0.190:43253)
0812 01:19:45.228575 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:45.228578 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:45.228642 (+ 64us) server_negotiation.cc:407] Connection header received
0812 01:19:45.228690 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:45.228691 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:45.228703 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:45.228752 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:45.229059 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:45.229568 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:45.230233 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:45.230419 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:45.230469 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:45.233296 (+ 2827us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.0.130:41701: BlockingRecv error: recv got EOF from 127.24.0.130:41701 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 12 01:19:45 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.131@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:19:45 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.131@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 01:19:45.359572 7059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:45.354521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:43253 (local address 127.24.0.131:38007)
0812 01:19:45.354725 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:45.354738 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:45.354806 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:45.354994 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:45.354996 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:45.355207 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:45.355261 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:45.355263 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:45.356000 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:45.356001 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:45.356633 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:45.356637 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:45.357069 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:45.357070 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:45.357137 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:45.357657 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:45.357663 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:45.359474 (+ 1811us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.0.190:43253: Server oryx/127.24.0.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":90,"threads_started":1}
W20260812 01:19:45.359625 7057 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:19:45.354662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.131:38007 (local address 127.24.0.190:43253)
0812 01:19:45.354737 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:45.354739 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:45.354801 (+ 62us) server_negotiation.cc:407] Connection header received
0812 01:19:45.354875 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:45.354876 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:45.354889 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:45.354940 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:45.355330 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:45.355907 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:45.356726 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:45.356989 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:45.357069 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:45.359558 (+ 2489us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.0.131:38007: BlockingRecv error: recv got EOF from 127.24.0.131:38007 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 12 01:19:46 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.129@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:19:46 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.129@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 01:19:46.126760 7060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:46.117921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:43253 (local address 127.24.0.129:34139)
0812 01:19:46.118046 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:46.118059 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:46.118128 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:46.118260 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:46.118261 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:46.118430 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:46.118481 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:46.118482 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:46.119139 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:46.119139 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:46.119652 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:46.119653 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:46.120039 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:46.120039 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:46.120104 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:46.124612 (+ 4508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:46.124617 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:46.126654 (+ 2037us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.0.190:43253: Server oryx/127.24.0.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":63,"threads_started":1}
W20260812 01:19:46.126827 7061 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:19:46.117991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.129:34139 (local address 127.24.0.190:43253)
0812 01:19:46.118138 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:46.118140 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:46.118148 (+ 8us) server_negotiation.cc:407] Connection header received
0812 01:19:46.118171 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:46.118171 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:46.118182 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:46.118241 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:46.118546 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:46.119073 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:46.119751 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:46.119953 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:46.120037 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:46.126750 (+ 6713us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.0.129:34139: BlockingRecv error: recv got EOF from 127.24.0.129:34139 (error 108)
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":74,"threads_started":1}
Aug 12 01:19:46 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.130@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:19:46 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.130@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 01:19:46.238947 7063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:46.233790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:43253 (local address 127.24.0.130:43195)
0812 01:19:46.234010 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:46.234022 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:46.234129 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:46.234303 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:46.234304 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:46.234502 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:46.234557 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:46.234559 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:46.235450 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:46.235451 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:46.236051 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:46.236053 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:46.236493 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:46.236494 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:46.236555 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:46.236999 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:46.237004 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:46.238835 (+ 1831us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.0.190:43253: Server oryx/127.24.0.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":104,"threads_started":1}
W20260812 01:19:46.238988 7061 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:19:46.233917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.130:43195 (local address 127.24.0.190:43253)
0812 01:19:46.233992 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:46.233995 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:46.234108 (+ 113us) server_negotiation.cc:407] Connection header received
0812 01:19:46.234191 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:46.234192 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:46.234208 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:46.234279 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:46.234646 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:46.235356 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:46.236180 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:46.236401 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:46.236492 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:46.238923 (+ 2431us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.0.130:43195: BlockingRecv error: recv got EOF from 127.24.0.130:43195 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 12 01:19:46 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.131@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:19:46 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.131@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 01:19:46.364660 7064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:46.360159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:43253 (local address 127.24.0.131:51507)
0812 01:19:46.360327 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:46.360337 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:46.360392 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:46.360535 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:46.360536 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:46.360710 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:46.360760 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:46.360761 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:46.361407 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:46.361408 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:46.361965 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:46.361966 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:46.362323 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:46.362324 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:46.362363 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:46.362817 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:46.362821 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:46.364588 (+ 1767us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.0.190:43253: Server oryx/127.24.0.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":104,"threads_started":1}
W20260812 01:19:46.364781 7061 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:19:46.360285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.131:51507 (local address 127.24.0.190:43253)
0812 01:19:46.360334 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:46.360337 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:46.360389 (+ 52us) server_negotiation.cc:407] Connection header received
0812 01:19:46.360437 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:46.360437 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:46.360449 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:46.360505 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:46.360827 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:46.361334 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:46.362048 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:46.362252 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:46.362310 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:46.364683 (+ 2373us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.0.131:51507: BlockingRecv error: recv got EOF from 127.24.0.131:51507 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
Aug 12 01:19:47 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.129@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:19:47 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.129@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 01:19:47.132393 7065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:47.127302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:43253 (local address 127.24.0.129:35723)
0812 01:19:47.127468 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:47.127479 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:47.127539 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:47.127684 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:47.127685 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:47.127863 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:47.127951 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:47.127953 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:47.128895 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:47.128896 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:47.129437 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:47.129439 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:47.129773 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:47.129774 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:47.129816 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:47.130278 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:47.130282 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:47.132281 (+ 1999us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.0.190:43253: Server oryx/127.24.0.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":97,"threads_started":1}
W20260812 01:19:47.132493 7066 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:19:47.127421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.129:35723 (local address 127.24.0.190:43253)
0812 01:19:47.127531 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:47.127532 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:47.127542 (+ 10us) server_negotiation.cc:407] Connection header received
0812 01:19:47.127591 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:47.127592 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:47.127603 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:47.127664 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:47.128121 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:47.128805 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:47.129509 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:47.129707 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:47.129758 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:47.132415 (+ 2657us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.0.129:35723: BlockingRecv error: recv got EOF from 127.24.0.129:35723 (error 108)
Metrics: {"server-negotiator.queue_time_us":83,"thread_start_us":54,"threads_started":1}
Aug 12 01:19:47 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.130@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:19:47 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.130@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 01:19:47.244046 7067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:47.239562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:43253 (local address 127.24.0.130:56581)
0812 01:19:47.239708 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:47.239722 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:47.239792 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:47.239926 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:47.239926 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:47.240127 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:47.240175 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:47.240177 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:47.240806 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:47.240807 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:47.241316 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:47.241317 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:47.241698 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:47.241699 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:47.241747 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:47.242232 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:47.242237 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:47.243975 (+ 1738us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.0.190:43253: Server oryx/127.24.0.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":71,"threads_started":1}
W20260812 01:19:47.244230 7066 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:19:47.239654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.130:56581 (local address 127.24.0.190:43253)
0812 01:19:47.239741 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:47.239743 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:47.239788 (+ 45us) server_negotiation.cc:407] Connection header received
0812 01:19:47.239849 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:47.239850 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:47.239862 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:47.239904 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:47.240241 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:47.240738 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:47.241410 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:47.241618 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:47.241685 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:47.244161 (+ 2476us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.0.130:56581: BlockingRecv error: recv got EOF from 127.24.0.130:56581 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 12 01:19:47 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.131@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:19:47 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.131@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 01:19:47.370042 7070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:47.365255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:43253 (local address 127.24.0.131:46443)
0812 01:19:47.365439 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:47.365458 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:47.365541 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:47.365761 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:47.365762 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:47.366001 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:47.366059 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:47.366061 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:47.366714 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:47.366715 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:47.367260 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:47.367262 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:47.367612 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:47.367612 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:47.367654 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:47.368145 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:47.368151 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:47.369970 (+ 1819us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.0.190:43253: Server oryx/127.24.0.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":97,"threads_started":1}
W20260812 01:19:47.370208 7066 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:19:47.365364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.131:46443 (local address 127.24.0.190:43253)
0812 01:19:47.365437 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:47.365439 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:47.365544 (+ 105us) server_negotiation.cc:407] Connection header received
0812 01:19:47.365643 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:47.365644 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:47.365660 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:47.365749 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:47.366134 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:47.366636 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:47.367348 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:47.367539 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:47.367609 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:47.370114 (+ 2505us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.0.131:46443: BlockingRecv error: recv got EOF from 127.24.0.131:46443 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 12 01:19:48 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.129@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:19:48 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.129@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 01:19:48.137529 7071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:48.132935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:43253 (local address 127.24.0.129:35101)
0812 01:19:48.133116 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:48.133128 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:48.133204 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:48.133351 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:48.133352 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:48.133531 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:48.133580 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:48.133581 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:48.134241 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:48.134242 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:48.134761 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:48.134763 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:48.135138 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:48.135139 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:48.135187 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:48.135633 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:48.135637 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:48.137463 (+ 1826us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.0.190:43253: Server oryx/127.24.0.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":121,"threads_started":1}
W20260812 01:19:48.137611 7073 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:19:48.133043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.129:35101 (local address 127.24.0.190:43253)
0812 01:19:48.133186 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:48.133188 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:48.133197 (+ 9us) server_negotiation.cc:407] Connection header received
0812 01:19:48.133254 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:48.133255 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:48.133266 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:48.133329 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:48.133649 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:48.134172 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:48.134847 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:48.135047 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:48.135125 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:48.137557 (+ 2432us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.0.129:35101: BlockingRecv error: recv got EOF from 127.24.0.129:35101 (error 108)
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":82,"threads_started":1}
Aug 12 01:19:48 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.130@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:19:48 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.130@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 01:19:48.249272 7074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:48.244649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:43253 (local address 127.24.0.130:43455)
0812 01:19:48.244819 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:48.244830 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:48.244883 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:48.245014 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:48.245015 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:48.245197 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:48.245245 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:48.245246 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:48.245905 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:48.245906 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:48.246450 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:48.246452 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:48.246814 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:48.246816 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:48.246859 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:48.247354 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:48.247359 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:48.249211 (+ 1852us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.0.190:43253: Server oryx/127.24.0.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":95,"threads_started":1}
W20260812 01:19:48.249439 7073 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:19:48.244756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.130:43455 (local address 127.24.0.190:43253)
0812 01:19:48.244861 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:48.244863 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:48.244884 (+ 21us) server_negotiation.cc:407] Connection header received
0812 01:19:48.244950 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:48.244951 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:48.244962 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:48.245002 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:48.245309 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:48.245815 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:48.246527 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:48.246722 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:48.246774 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:48.249335 (+ 2561us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.0.130:43455: BlockingRecv error: recv got EOF from 127.24.0.130:43455 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Aug 12 01:19:48 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.131@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:19:48 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.131@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 01:19:48.375228 7075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:48.370494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:43253 (local address 127.24.0.131:38485)
0812 01:19:48.370633 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:48.370644 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:48.370703 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:48.370831 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:48.370832 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:48.371006 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:48.371055 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:48.371056 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:48.371805 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:48.371806 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:48.372457 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:48.372461 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:48.372831 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:48.372831 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:48.372872 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:48.373333 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:48.373338 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:48.375149 (+ 1811us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.0.190:43253: Server oryx/127.24.0.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":82,"threads_started":1}
W20260812 01:19:48.375296 7073 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:19:48.370561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.131:38485 (local address 127.24.0.190:43253)
0812 01:19:48.370605 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:48.370607 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:48.370688 (+ 81us) server_negotiation.cc:407] Connection header received
0812 01:19:48.370747 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:48.370747 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:48.370759 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:48.370815 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:48.371117 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:48.371733 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:48.372537 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:48.372727 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:48.372832 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:48.375246 (+ 2414us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.0.131:38485: BlockingRecv error: recv got EOF from 127.24.0.131:38485 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
Aug 12 01:19:49 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.129@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:19:49 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.129@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 01:19:49.142673 7076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:49.138069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:43253 (local address 127.24.0.129:47553)
0812 01:19:49.138219 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:49.138231 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:49.138293 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:49.138440 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:49.138441 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:49.138622 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:49.138670 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:49.138671 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:49.139305 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:49.139306 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:49.139857 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:49.139859 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:49.140202 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:49.140203 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:49.140248 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:49.140694 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:49.140699 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:49.142558 (+ 1859us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.0.190:43253: Server oryx/127.24.0.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":95,"threads_started":1}
W20260812 01:19:49.142719 7077 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:19:49.138147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.129:47553 (local address 127.24.0.190:43253)
0812 01:19:49.138282 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:49.138288 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:49.138297 (+ 9us) server_negotiation.cc:407] Connection header received
0812 01:19:49.138357 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:49.138358 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:49.138369 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:49.138421 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:49.138735 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:49.139234 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:49.139926 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:49.140133 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:49.140199 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:49.142639 (+ 2440us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.0.129:47553: BlockingRecv error: recv got EOF from 127.24.0.129:47553 (error 108)
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":78,"threads_started":1}
Aug 12 01:19:49 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.130@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:19:49 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.130@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 01:19:49.254958 7081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:49.249831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:43253 (local address 127.24.0.130:58105)
0812 01:19:49.250037 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:49.250048 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:49.250159 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:49.250361 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:49.250363 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:49.250564 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:49.250611 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:49.250612 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:49.251321 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:49.251322 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:49.251877 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:49.251878 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:49.252361 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:49.252362 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:49.252436 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:49.252956 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:49.252961 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:49.254845 (+ 1884us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.0.190:43253: Server oryx/127.24.0.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":119,"threads_started":1}
W20260812 01:19:49.254997 7077 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:19:49.250003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.130:58105 (local address 127.24.0.190:43253)
0812 01:19:49.250134 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:49.250136 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:49.250148 (+ 12us) server_negotiation.cc:407] Connection header received
0812 01:19:49.250255 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:49.250257 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:49.250270 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:49.250356 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:49.250704 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:49.251216 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:49.251996 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:49.252279 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:49.252364 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:49.254931 (+ 2567us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.0.130:58105: BlockingRecv error: recv got EOF from 127.24.0.130:58105 (error 108)
Metrics: {"server-negotiator.queue_time_us":101}
Aug 12 01:19:49 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.131@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:19:49 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.131@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 01:19:49.380682 7084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:49.375801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:43253 (local address 127.24.0.131:32825)
0812 01:19:49.375957 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:49.375968 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:49.376030 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:49.376209 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:49.376210 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:49.376390 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:49.376450 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:49.376452 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:49.377095 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:49.377096 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:49.377655 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:49.377656 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:49.378023 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:49.378025 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:49.378103 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:49.378660 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:49.378665 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:49.380623 (+ 1958us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.0.190:43253: Server oryx/127.24.0.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":92,"threads_started":1}
W20260812 01:19:49.380785 7077 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:19:49.375877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.131:32825 (local address 127.24.0.190:43253)
0812 01:19:49.375959 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:49.375961 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:49.376032 (+ 71us) server_negotiation.cc:407] Connection header received
0812 01:19:49.376110 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:49.376110 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:49.376126 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:49.376191 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:49.376529 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:49.377019 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:49.377741 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:49.377924 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:49.377977 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:49.380713 (+ 2736us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.0.131:32825: BlockingRecv error: recv got EOF from 127.24.0.131:32825 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 12 01:19:50 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.129@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:19:50 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.129@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 01:19:50.148475 7086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:50.143343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:43253 (local address 127.24.0.129:47617)
0812 01:19:50.143507 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:50.143529 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:50.143579 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:50.143753 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:50.143754 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:50.143943 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:50.143993 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:50.143995 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:50.144708 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:50.144709 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:50.145239 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:50.145241 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:50.145613 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:50.145613 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:50.145690 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:50.146382 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:50.146388 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:50.148399 (+ 2011us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.0.190:43253: Server oryx/127.24.0.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":97,"threads_started":1}
W20260812 01:19:50.148586 7087 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:19:50.143461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.129:47617 (local address 127.24.0.190:43253)
0812 01:19:50.143604 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:50.143606 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:50.143615 (+ 9us) server_negotiation.cc:407] Connection header received
0812 01:19:50.143649 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:50.143650 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:50.143661 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:50.143735 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:50.144102 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:50.144632 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:50.145337 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:50.145541 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:50.145604 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:50.148502 (+ 2898us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.0.129:47617: BlockingRecv error: recv got EOF from 127.24.0.129:47617 (error 108)
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":87,"threads_started":1}
Aug 12 01:19:50 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.130@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:19:50 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.130@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 01:19:50.260383 7089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:50.255540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:43253 (local address 127.24.0.130:48957)
0812 01:19:50.255696 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:50.255715 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:50.255770 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:50.255890 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:50.255890 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:50.256161 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:50.256213 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:50.256215 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:50.256870 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:50.256871 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:50.257415 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:50.257417 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:50.257797 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:50.257798 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:50.257865 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:50.258455 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:50.258460 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:50.260309 (+ 1849us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.0.190:43253: Server oryx/127.24.0.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":75,"threads_started":1}
W20260812 01:19:50.260501 7087 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:19:50.255650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.130:48957 (local address 127.24.0.190:43253)
0812 01:19:50.255737 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:50.255739 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:50.255752 (+ 13us) server_negotiation.cc:407] Connection header received
0812 01:19:50.255830 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:50.255831 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:50.255842 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:50.255883 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:50.256297 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:50.256797 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:50.257505 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:50.257705 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:50.257762 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:50.260404 (+ 2642us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.0.130:48957: BlockingRecv error: recv got EOF from 127.24.0.130:48957 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 12 01:19:50 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.131@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:19:50 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.131@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 01:19:50.386245 7090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:50.381217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:43253 (local address 127.24.0.131:47307)
0812 01:19:50.381391 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:50.381404 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:50.381503 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:50.381689 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:50.381691 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:50.381889 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:50.381935 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:50.381937 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:50.382649 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:50.382650 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:50.383181 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:50.383183 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:50.383603 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:50.383604 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:50.383667 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:50.384274 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:50.384281 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:50.386171 (+ 1890us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.0.190:43253: Server oryx/127.24.0.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":91,"threads_started":1}
W20260812 01:19:50.386384 7087 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:19:50.381304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.131:47307 (local address 127.24.0.190:43253)
0812 01:19:50.381395 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:50.381398 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:50.381480 (+ 82us) server_negotiation.cc:407] Connection header received
0812 01:19:50.381573 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:50.381574 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:50.381588 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:50.381678 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:50.382024 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:50.382550 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:50.383271 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:50.383524 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:50.383604 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:50.386273 (+ 2669us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.0.131:47307: BlockingRecv error: recv got EOF from 127.24.0.131:47307 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 12 01:19:51 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.129@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:19:51 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.129@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 01:19:51.160857 7092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:51.149119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:43253 (local address 127.24.0.129:43419)
0812 01:19:51.149271 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:51.149286 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:51.149346 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:51.149492 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:51.149492 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:51.149665 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:51.149717 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:51.149719 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:51.150415 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:51.150416 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:51.150957 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:51.150959 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:51.151334 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:51.151335 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:51.151386 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:51.158373 (+ 6987us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:51.158380 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:51.160792 (+ 2412us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.0.190:43253: Server oryx/127.24.0.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":86,"threads_started":1}
W20260812 01:19:51.161080 7095 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:19:51.149184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.129:43419 (local address 127.24.0.190:43253)
0812 01:19:51.149311 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:51.149313 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:51.149326 (+ 13us) server_negotiation.cc:407] Connection header received
0812 01:19:51.149419 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:51.149420 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:51.149431 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:51.149487 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:51.149795 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:51.150330 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:51.151038 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:51.151243 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:51.151296 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:51.160992 (+ 9696us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.0.129:43419: BlockingRecv error: recv got EOF from 127.24.0.129:43419 (error 108)
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":66,"threads_started":1}
Aug 12 01:19:51 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.130@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:19:51 dist-test-slave-t3q3 krb5kdc[5353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.0.130@KRBTEST.COM for oryx/127.24.0.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 01:19:51.265733 7096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:51.260901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:43253 (local address 127.24.0.130:51613)
0812 01:19:51.261217 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:51.261227 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:51.261285 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:51.261414 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:51.261415 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:51.261585 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:51.261637 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:51.261639 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:51.262278 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:51.262279 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:51.262790 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:51.262791 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:51.263129 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:51.263130 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:51.263177 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:51.263709 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:51.263722 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:51.265673 (+ 1951us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.0.190:43253: Server oryx/127.24.0.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":198,"threads_started":1}
W20260812 01:19:51.265878 7095 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:19:51.261150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.130:51613 (local address 127.24.0.190:43253)
0812 01:19:51.261218 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:51.261220 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:51.261284 (+ 64us) server_negotiation.cc:407] Connection header received
0812 01:19:51.261351 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:51.261352 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:51.261365 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:51.261416 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:51.261697 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:51.262208 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:51.262862 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:51.263060 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:51.263114 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:51.265792 (+ 2678us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.0.130:51613: BlockingRecv error: recv got EOF from 127.24.0.130:51613 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 01:19:51.311887 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 6303
I20260812 01:19:51.328333 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 6565
I20260812 01:19:51.337819 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 6785
I20260812 01:19:51.345682 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 6187
2026-08-12T01:19:51Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17795 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-12T01:19:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:19:51Z Disabled control of system clock
I20260812 01:19:51.382970 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786497534195564-24578-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786497534195564-24578-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786497534195564-24578-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.190:43479
--webserver_interface=127.24.0.190
--webserver_port=0
--builtin_ntp_servers=127.24.0.148:44489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.0.190:43479
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260812 01:19:51.463626 7101 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:19:51.463815 7101 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:19:51.463836 7101 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:19:51.465014 7101 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:19:51.465057 7101 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:19:51.465070 7101 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:19:51.465083 7101 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:19:51.465095 7101 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:19:51.466485 7101 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:44489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786497534195564-24578-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786497534195564-24578-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.0.190:43479
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.190:43479
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.0.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7101
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786497534195564-24578-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:19:51.466729 7101 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:19:51.467048 7101 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 01:19:51.472175 7101 server_base.cc:1034] running on GCE node
W20260812 01:19:51.472162 7117 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:51.472177 7111 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:51.472376 7113 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:19:51.472643 7101 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:19:51.472945 7101 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:19:51.474143 7101 hybrid_clock.cc:648] HybridClock initialized: now 1786497591474120 us; error 39 us; skew 500 ppm
I20260812 01:19:51.475391 7101 webserver.cc:460] Webserver started at http://127.24.0.190:35979/ using document root <none> and password file <none>
I20260812 01:19:51.475653 7101 fs_manager.cc:359] Metadata directory not provided
I20260812 01:19:51.475724 7101 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:19:51.475875 7101 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:19:51.476964 7101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786497534195564-24578-0/minicluster-data/master-0/data/instance:
uuid: "fb6f8577bdba4a57bf72c3aad29594d2"
format_stamp: "Formatted at 2026-08-12 01:19:51 on dist-test-slave-t3q3"
server_key: "504f66155018e87bb1f4f4cf1e3b4afe"
server_key_iv: "bb5a6154d3ef02a587cb6d299e034e7e"
server_key_version: "encryptionkey@0"
I20260812 01:19:51.477344 7101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786497534195564-24578-0/minicluster-data/master-0/wal/instance:
uuid: "fb6f8577bdba4a57bf72c3aad29594d2"
format_stamp: "Formatted at 2026-08-12 01:19:51 on dist-test-slave-t3q3"
server_key: "504f66155018e87bb1f4f4cf1e3b4afe"
server_key_iv: "bb5a6154d3ef02a587cb6d299e034e7e"
server_key_version: "encryptionkey@0"
I20260812 01:19:51.478585 7101 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 01:19:51.479264 7130 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:51.479432 7101 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 01:19:51.479542 7101 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786497534195564-24578-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786497534195564-24578-0/minicluster-data/master-0/wal
uuid: "fb6f8577bdba4a57bf72c3aad29594d2"
format_stamp: "Formatted at 2026-08-12 01:19:51 on dist-test-slave-t3q3"
server_key: "504f66155018e87bb1f4f4cf1e3b4afe"
server_key_iv: "bb5a6154d3ef02a587cb6d299e034e7e"
server_key_version: "encryptionkey@0"
I20260812 01:19:51.479616 7101 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786497534195564-24578-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786497534195564-24578-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786497534195564-24578-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:19:51.485808 7101 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:19:51.486063 7101 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:19:51.486205 7101 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:19:51.490288 7101 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.190:43479
I20260812 01:19:51.490324 7218 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.190:43479 every 8 connection(s)
I20260812 01:19:51.490674 7101 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
I20260812 01:19:51.491372 7219 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:19:51.493990 7219 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb6f8577bdba4a57bf72c3aad29594d2: Bootstrap starting.
I20260812 01:19:51.494568 7219 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fb6f8577bdba4a57bf72c3aad29594d2: Neither blocks nor log segments found. Creating new log.
I20260812 01:19:51.494807 7219 log.cc:824] T 00000000000000000000000000000000 P fb6f8577bdba4a57bf72c3aad29594d2: Log is configured to *not* fsync() on all Append() calls
I20260812 01:19:51.495491 7219 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb6f8577bdba4a57bf72c3aad29594d2: No bootstrap required, opened a new log
I20260812 01:19:51.497139 7219 raft_consensus.cc:348] T 00000000000000000000000000000000 P fb6f8577bdba4a57bf72c3aad29594d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb6f8577bdba4a57bf72c3aad29594d2" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 43479 } }
I20260812 01:19:51.497273 7219 raft_consensus.cc:374] T 00000000000000000000000000000000 P fb6f8577bdba4a57bf72c3aad29594d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:19:51.497298 7219 raft_consensus.cc:729] T 00000000000000000000000000000000 P fb6f8577bdba4a57bf72c3aad29594d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb6f8577bdba4a57bf72c3aad29594d2, State: Initialized, Role: FOLLOWER
I20260812 01:19:51.497432 7219 consensus_queue.cc:260] T 00000000000000000000000000000000 P fb6f8577bdba4a57bf72c3aad29594d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb6f8577bdba4a57bf72c3aad29594d2" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 43479 } }
I20260812 01:19:51.497535 7219 raft_consensus.cc:388] T 00000000000000000000000000000000 P fb6f8577bdba4a57bf72c3aad29594d2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:19:51.497584 7219 raft_consensus.cc:482] T 00000000000000000000000000000000 P fb6f8577bdba4a57bf72c3aad29594d2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:19:51.497641 7219 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fb6f8577bdba4a57bf72c3aad29594d2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:19:51.497648 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 7101
I20260812 01:19:51.497738 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786497534195564-24578-0/minicluster-data/master-0/wal/instance
I20260812 01:19:51.497951 24578 external_mini_cluster.cc:1428] Setting key 7a654c3f7a32c2519bdedee5341160d4
I20260812 01:19:51.498363 7219 raft_consensus.cc:504] T 00000000000000000000000000000000 P fb6f8577bdba4a57bf72c3aad29594d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb6f8577bdba4a57bf72c3aad29594d2" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 43479 } }
I20260812 01:19:51.498554 7219 leader_election.cc:302] T 00000000000000000000000000000000 P fb6f8577bdba4a57bf72c3aad29594d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fb6f8577bdba4a57bf72c3aad29594d2; no voters:
I20260812 01:19:51.498788 7219 leader_election.cc:288] T 00000000000000000000000000000000 P fb6f8577bdba4a57bf72c3aad29594d2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 01:19:51.498813 7229 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fb6f8577bdba4a57bf72c3aad29594d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:19:51.498927 7229 raft_consensus.cc:686] T 00000000000000000000000000000000 P fb6f8577bdba4a57bf72c3aad29594d2 [term 1 LEADER]: Becoming Leader. State: Replica: fb6f8577bdba4a57bf72c3aad29594d2, State: Running, Role: LEADER
I20260812 01:19:51.499051 7219 sys_catalog.cc:564] T 00000000000000000000000000000000 P fb6f8577bdba4a57bf72c3aad29594d2 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:19:51.499010 7229 consensus_queue.cc:237] T 00000000000000000000000000000000 P fb6f8577bdba4a57bf72c3aad29594d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb6f8577bdba4a57bf72c3aad29594d2" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 43479 } }
I20260812 01:19:51.499907 7232 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb6f8577bdba4a57bf72c3aad29594d2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fb6f8577bdba4a57bf72c3aad29594d2. Latest consensus state: current_term: 1 leader_uuid: "fb6f8577bdba4a57bf72c3aad29594d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb6f8577bdba4a57bf72c3aad29594d2" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 43479 } } }
I20260812 01:19:51.500000 7232 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb6f8577bdba4a57bf72c3aad29594d2 [sys.catalog]: This master's current role is: LEADER
I20260812 01:19:51.500453 7230 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb6f8577bdba4a57bf72c3aad29594d2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fb6f8577bdba4a57bf72c3aad29594d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb6f8577bdba4a57bf72c3aad29594d2" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 43479 } } }
I20260812 01:19:51.500538 7230 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb6f8577bdba4a57bf72c3aad29594d2 [sys.catalog]: This master's current role is: LEADER
W20260812 01:19:51.501075 7258 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fb6f8577bdba4a57bf72c3aad29594d2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 01:19:51.501145 7258 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 01:19:51.501381 7244 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:19:51.501583 7244 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:19:51.503240 7244 catalog_manager.cc:1347] Generated new cluster ID: 7cb4c12bda7d49f88e3ba270590e0650
I20260812 01:19:51.503300 7244 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 01:19:51.505136 7227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:51.498463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48688 (local address 127.24.0.190:43479)
0812 01:19:51.498674 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:51.498682 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:51.498722 (+ 40us) server_negotiation.cc:407] Connection header received
0812 01:19:51.498814 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:51.498819 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:51.498905 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:51.499147 (+ 242us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:51.499364 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:51.501211 (+ 1847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:51.502152 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:51.502641 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:51.502771 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:51.504308 (+ 1537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:51.504312 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:51.504312 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:19:51.504523 (+ 211us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:51.504567 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:19:51.504581 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:51.504659 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:51.504728 (+ 69us) server_negotiation.cc:299] Negotiation successful
0812 01:19:51.504915 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":98,"threads_started":1}
I20260812 01:19:51.516515 7244 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:19:51.517052 7244 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:19:51.525031 7244 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fb6f8577bdba4a57bf72c3aad29594d2: Generated new TSK 0
I20260812 01:19:51.525242 7244 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 01:19:51.528102 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786497534195564-24578-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786497534195564-24578-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786497534195564-24578-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786497534195564-24578-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.0.129:0
--local_ip_for_outbound_sockets=127.24.0.129
--webserver_interface=127.24.0.129
--webserver_port=0
--tserver_master_addrs=127.24.0.190:43479
--builtin_ntp_servers=127.24.0.148:44489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260812 01:19:51.635070 7264 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:19:51.635277 7264 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:19:51.635299 7264 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:19:51.636538 7264 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:19:51.636577 7264 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:19:51.636602 7264 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.0.129
I20260812 01:19:51.637924 7264 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:44489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786497534195564-24578-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786497534195564-24578-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.0.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786497534195564-24578-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.0.129
--webserver_port=0
--tserver_master_addrs=127.24.0.190:43479
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7264
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.0.129
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786497534195564-24578-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:19:51.638182 7264 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:19:51.638506 7264 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:19:51.644407 7272 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:19:51.644615 7264 server_base.cc:1034] running on GCE node
W20260812 01:19:51.644716 7274 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:51.644733 7271 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:19:51.644973 7264 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:19:51.645237 7264 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:19:51.646404 7264 hybrid_clock.cc:648] HybridClock initialized: now 1786497591646390 us; error 31 us; skew 500 ppm
I20260812 01:19:51.647859 7264 webserver.cc:460] Webserver started at http://127.24.0.129:41765/ using document root <none> and password file <none>
I20260812 01:19:51.648171 7264 fs_manager.cc:359] Metadata directory not provided
I20260812 01:19:51.648247 7264 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:19:51.648391 7264 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:19:51.649468 7264 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786497534195564-24578-0/minicluster-data/ts-0/data/instance:
uuid: "a1016b2459e5458ebb24e3042f25542c"
format_stamp: "Formatted at 2026-08-12 01:19:51 on dist-test-slave-t3q3"
server_key: "69ea9f4a07d0926a15075712ff1c68cd"
server_key_iv: "d34fdaec446e3e581d58ae79e936d25c"
server_key_version: "encryptionkey@0"
I20260812 01:19:51.649857 7264 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786497534195564-24578-0/minicluster-data/ts-0/wal/instance:
uuid: "a1016b2459e5458ebb24e3042f25542c"
format_stamp: "Formatted at 2026-08-12 01:19:51 on dist-test-slave-t3q3"
server_key: "69ea9f4a07d0926a15075712ff1c68cd"
server_key_iv: "d34fdaec446e3e581d58ae79e936d25c"
server_key_version: "encryptionkey@0"
I20260812 01:19:51.651250 7264 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 01:19:51.652073 7281 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:51.652257 7264 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:19:51.652364 7264 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786497534195564-24578-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786497534195564-24578-0/minicluster-data/ts-0/wal
uuid: "a1016b2459e5458ebb24e3042f25542c"
format_stamp: "Formatted at 2026-08-12 01:19:51 on dist-test-slave-t3q3"
server_key: "69ea9f4a07d0926a15075712ff1c68cd"
server_key_iv: "d34fdaec446e3e581d58ae79e936d25c"
server_key_version: "encryptionkey@0"
I20260812 01:19:51.652454 7264 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786497534195564-24578-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786497534195564-24578-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786497534195564-24578-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:19:51.665735 7264 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:19:51.666064 7264 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:19:51.666229 7264 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:19:51.666468 7264 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:19:51.666822 7264 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:19:51.666879 7264 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:51.666923 7264 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:19:51.666947 7264 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:51.673384 7264 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.129:37323
I20260812 01:19:51.673445 7477 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.129:37323 every 8 connection(s)
I20260812 01:19:51.673779 7264 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786497534195564-24578-0/minicluster-data/ts-0/data/info.pb
I20260812 01:19:51.674786 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 7264
I20260812 01:19:51.674901 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786497534195564-24578-0/minicluster-data/ts-0/wal/instance
I20260812 01:19:51.675139 24578 external_mini_cluster.cc:1428] Setting key 43c0b5602dfab8403f2d7d38d53642e7
I20260812 01:19:51.676110 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786497534195564-24578-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786497534195564-24578-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786497534195564-24578-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786497534195564-24578-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.0.130:0
--local_ip_for_outbound_sockets=127.24.0.130
--webserver_interface=127.24.0.130
--webserver_port=0
--tserver_master_addrs=127.24.0.190:43479
--builtin_ntp_servers=127.24.0.148:44489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260812 01:19:51.679760 7227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:51.674530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.129:49081 (local address 127.24.0.190:43479)
0812 01:19:51.674633 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:51.674635 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:51.674845 (+ 210us) server_negotiation.cc:407] Connection header received
0812 01:19:51.675478 (+ 633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:51.675479 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:51.675495 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:51.675549 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:51.676348 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:51.676872 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:51.678156 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:51.678367 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:51.678434 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:51.679467 (+ 1033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:51.679468 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:51.679468 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:19:51.679476 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:51.679492 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:19:51.679496 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:51.679561 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:51.679666 (+ 105us) server_negotiation.cc:299] Negotiation successful
0812 01:19:51.679701 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 01:19:51.680033 7481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:51.674503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:43479 (local address 127.24.0.129:49081)
0812 01:19:51.674703 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:51.674744 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:51.675369 (+ 625us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:51.675622 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:51.675627 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:51.675638 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:51.676265 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:51.676274 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:51.676975 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:51.676977 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:51.678047 (+ 1070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:51.678051 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:51.678461 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:51.678463 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:51.678632 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:51.679275 (+ 643us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 01:19:51.679276 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:51.679317 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 01:19:51.679347 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:19:51.679575 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:19:51.679581 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:19:51.679585 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 01:19:51.679655 (+ 70us) client_negotiation.cc:240] Negotiation successful
0812 01:19:51.679772 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":92,"threads_started":1}
I20260812 01:19:51.680557 7478 heartbeater.cc:347] Connected to a master server at 127.24.0.190:43479
I20260812 01:19:51.680706 7478 heartbeater.cc:464] Registering TS with master...
I20260812 01:19:51.680925 7478 heartbeater.cc:511] Master 127.24.0.190:43479 requested a full tablet report, sending...
I20260812 01:19:51.681488 7151 ts_manager.cc:194] Registered new tserver with Master: a1016b2459e5458ebb24e3042f25542c (127.24.0.129:37323)
I20260812 01:19:51.682178 7151 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.0.129:49081
W20260812 01:19:51.759249 7482 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:19:51.759454 7482 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:19:51.759475 7482 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:19:51.760645 7482 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:19:51.760691 7482 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:19:51.760716 7482 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.0.130
I20260812 01:19:51.762008 7482 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:44489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786497534195564-24578-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786497534195564-24578-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.0.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786497534195564-24578-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.0.130
--webserver_port=0
--tserver_master_addrs=127.24.0.190:43479
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7482
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.0.130
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786497534195564-24578-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:19:51.762199 7482 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:19:51.762444 7482 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:19:51.767382 7497 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:51.767347 7499 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:51.767365 7496 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:19:51.767495 7482 server_base.cc:1034] running on GCE node
I20260812 01:19:51.767783 7482 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:19:51.768029 7482 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:19:51.769212 7482 hybrid_clock.cc:648] HybridClock initialized: now 1786497591769178 us; error 51 us; skew 500 ppm
I20260812 01:19:51.770442 7482 webserver.cc:460] Webserver started at http://127.24.0.130:41833/ using document root <none> and password file <none>
I20260812 01:19:51.770731 7482 fs_manager.cc:359] Metadata directory not provided
I20260812 01:19:51.770802 7482 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:19:51.770938 7482 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:19:51.771951 7482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786497534195564-24578-0/minicluster-data/ts-1/data/instance:
uuid: "19a6de376e9d420db464722a759f4928"
format_stamp: "Formatted at 2026-08-12 01:19:51 on dist-test-slave-t3q3"
server_key: "129446db35c7fb862e09bd3126e25b60"
server_key_iv: "01d4a94522b114cbc5552889b262e6f0"
server_key_version: "encryptionkey@0"
I20260812 01:19:51.772342 7482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786497534195564-24578-0/minicluster-data/ts-1/wal/instance:
uuid: "19a6de376e9d420db464722a759f4928"
format_stamp: "Formatted at 2026-08-12 01:19:51 on dist-test-slave-t3q3"
server_key: "129446db35c7fb862e09bd3126e25b60"
server_key_iv: "01d4a94522b114cbc5552889b262e6f0"
server_key_version: "encryptionkey@0"
I20260812 01:19:51.773650 7482 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:19:51.774349 7510 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:51.774510 7482 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:51.774605 7482 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786497534195564-24578-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786497534195564-24578-0/minicluster-data/ts-1/wal
uuid: "19a6de376e9d420db464722a759f4928"
format_stamp: "Formatted at 2026-08-12 01:19:51 on dist-test-slave-t3q3"
server_key: "129446db35c7fb862e09bd3126e25b60"
server_key_iv: "01d4a94522b114cbc5552889b262e6f0"
server_key_version: "encryptionkey@0"
I20260812 01:19:51.774664 7482 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786497534195564-24578-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786497534195564-24578-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786497534195564-24578-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:19:51.787544 7482 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:19:51.787788 7482 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:19:51.787887 7482 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:19:51.788105 7482 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:19:51.788427 7482 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:19:51.788460 7482 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:51.788480 7482 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:19:51.788493 7482 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:51.793906 7482 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.130:34665
I20260812 01:19:51.794023 7703 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.130:34665 every 8 connection(s)
I20260812 01:19:51.794322 7482 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786497534195564-24578-0/minicluster-data/ts-1/data/info.pb
I20260812 01:19:51.799649 7227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:51.795058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.130:53257 (local address 127.24.0.190:43479)
0812 01:19:51.795133 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:51.795135 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:51.795376 (+ 241us) server_negotiation.cc:407] Connection header received
0812 01:19:51.795842 (+ 466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:51.795843 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:51.795853 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:51.795914 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:51.796671 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:51.797240 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:51.798178 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:51.798388 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:51.798454 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:51.799304 (+ 850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:51.799305 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:51.799305 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:19:51.799311 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:51.799329 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:19:51.799334 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:51.799393 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:51.799561 (+ 168us) server_negotiation.cc:299] Negotiation successful
0812 01:19:51.799589 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 01:19:51.799697 7706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:51.795029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:43479 (local address 127.24.0.130:53257)
0812 01:19:51.795248 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:51.795288 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:51.795756 (+ 468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:51.795983 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:51.795986 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:51.795995 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:51.796559 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:51.796564 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:51.797332 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:51.797333 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:51.798087 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:51.798089 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:51.798473 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:51.798474 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:51.798580 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:51.799099 (+ 519us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 01:19:51.799100 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:51.799127 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 01:19:51.799149 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:19:51.799405 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:19:51.799410 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:19:51.799412 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 01:19:51.799467 (+ 55us) client_negotiation.cc:240] Negotiation successful
0812 01:19:51.799545 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":109,"threads_started":1}
I20260812 01:19:51.800227 7704 heartbeater.cc:347] Connected to a master server at 127.24.0.190:43479
I20260812 01:19:51.800336 7704 heartbeater.cc:464] Registering TS with master...
I20260812 01:19:51.800526 7704 heartbeater.cc:511] Master 127.24.0.190:43479 requested a full tablet report, sending...
I20260812 01:19:51.800937 7151 ts_manager.cc:194] Registered new tserver with Master: 19a6de376e9d420db464722a759f4928 (127.24.0.130:34665)
I20260812 01:19:51.801327 7151 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.0.130:53257
I20260812 01:19:51.802387 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 7482
I20260812 01:19:51.802461 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786497534195564-24578-0/minicluster-data/ts-1/wal/instance
I20260812 01:19:51.802712 24578 external_mini_cluster.cc:1428] Setting key 38be6cf11fedd1ac0423971b0cc8714a
I20260812 01:19:51.803722 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786497534195564-24578-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786497534195564-24578-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786497534195564-24578-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786497534195564-24578-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.0.131:0
--local_ip_for_outbound_sockets=127.24.0.131
--webserver_interface=127.24.0.131
--webserver_port=0
--tserver_master_addrs=127.24.0.190:43479
--builtin_ntp_servers=127.24.0.148:44489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260812 01:19:51.887619 7708 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:19:51.887830 7708 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:19:51.887851 7708 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:19:51.889154 7708 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:19:51.889201 7708 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:19:51.889226 7708 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.0.131
I20260812 01:19:51.890573 7708 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:44489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786497534195564-24578-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786497534195564-24578-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.0.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786497534195564-24578-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.0.131
--webserver_port=0
--tserver_master_addrs=127.24.0.190:43479
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7708
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.0.131
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786497534195564-24578-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:19:51.890823 7708 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:19:51.891137 7708 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:19:51.896214 7716 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:51.896224 7718 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:51.896255 7715 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:19:51.896642 7708 server_base.cc:1034] running on GCE node
I20260812 01:19:51.896842 7708 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:19:51.897068 7708 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:19:51.898234 7708 hybrid_clock.cc:648] HybridClock initialized: now 1786497591898218 us; error 30 us; skew 500 ppm
I20260812 01:19:51.899734 7708 webserver.cc:460] Webserver started at http://127.24.0.131:39411/ using document root <none> and password file <none>
I20260812 01:19:51.899989 7708 fs_manager.cc:359] Metadata directory not provided
I20260812 01:19:51.900059 7708 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:19:51.900288 7708 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:19:51.901345 7708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786497534195564-24578-0/minicluster-data/ts-2/data/instance:
uuid: "9d40a51aa0524b72953c0b946a46da01"
format_stamp: "Formatted at 2026-08-12 01:19:51 on dist-test-slave-t3q3"
server_key: "8a30d59d110eed00f199088f64826578"
server_key_iv: "b041846c2e625e42e22471a1391a385c"
server_key_version: "encryptionkey@0"
I20260812 01:19:51.901716 7708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786497534195564-24578-0/minicluster-data/ts-2/wal/instance:
uuid: "9d40a51aa0524b72953c0b946a46da01"
format_stamp: "Formatted at 2026-08-12 01:19:51 on dist-test-slave-t3q3"
server_key: "8a30d59d110eed00f199088f64826578"
server_key_iv: "b041846c2e625e42e22471a1391a385c"
server_key_version: "encryptionkey@0"
I20260812 01:19:51.903092 7708 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 01:19:51.903855 7731 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:51.904031 7708 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 01:19:51.904165 7708 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786497534195564-24578-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786497534195564-24578-0/minicluster-data/ts-2/wal
uuid: "9d40a51aa0524b72953c0b946a46da01"
format_stamp: "Formatted at 2026-08-12 01:19:51 on dist-test-slave-t3q3"
server_key: "8a30d59d110eed00f199088f64826578"
server_key_iv: "b041846c2e625e42e22471a1391a385c"
server_key_version: "encryptionkey@0"
I20260812 01:19:51.904250 7708 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786497534195564-24578-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786497534195564-24578-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786497534195564-24578-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:19:51.932263 7708 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:19:51.932623 7708 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:19:51.932804 7708 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:19:51.933084 7708 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:19:51.933455 7708 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:19:51.933513 7708 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:51.933558 7708 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:19:51.933583 7708 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:51.939692 7708 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.131:44481
I20260812 01:19:51.939800 7923 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.131:44481 every 8 connection(s)
I20260812 01:19:51.940171 7708 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786497534195564-24578-0/minicluster-data/ts-2/data/info.pb
I20260812 01:19:51.945267 7227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:51.941005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.131:33535 (local address 127.24.0.190:43479)
0812 01:19:51.941094 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:51.941100 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:51.941292 (+ 192us) server_negotiation.cc:407] Connection header received
0812 01:19:51.941772 (+ 480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:51.941773 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:51.941783 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:51.941845 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:51.942523 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:51.943032 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:51.943977 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:51.944209 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:51.944269 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:51.945053 (+ 784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:51.945054 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:51.945054 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:19:51.945058 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:51.945074 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:19:51.945078 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:51.945122 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:51.945184 (+ 62us) server_negotiation.cc:299] Negotiation successful
0812 01:19:51.945209 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 01:19:51.945497 7927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:51.940950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:43479 (local address 127.24.0.131:33535)
0812 01:19:51.941148 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:51.941199 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:51.941688 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:51.941900 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:51.941903 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:51.941911 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:51.942410 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:51.942415 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:51.943120 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:51.943121 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:51.943889 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:51.943891 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:51.944286 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:51.944286 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:51.944397 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:51.944893 (+ 496us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 01:19:51.944894 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:51.944922 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 01:19:51.944944 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:19:51.945122 (+ 178us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:19:51.945124 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:19:51.945126 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 01:19:51.945178 (+ 52us) client_negotiation.cc:240] Negotiation successful
0812 01:19:51.945253 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":98,"threads_started":1}
I20260812 01:19:51.946084 7924 heartbeater.cc:347] Connected to a master server at 127.24.0.190:43479
I20260812 01:19:51.946165 7924 heartbeater.cc:464] Registering TS with master...
I20260812 01:19:51.946367 7924 heartbeater.cc:511] Master 127.24.0.190:43479 requested a full tablet report, sending...
I20260812 01:19:51.946785 7151 ts_manager.cc:194] Registered new tserver with Master: 9d40a51aa0524b72953c0b946a46da01 (127.24.0.131:44481)
I20260812 01:19:51.947184 7151 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.0.131:33535
I20260812 01:19:51.950085 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 7708
I20260812 01:19:51.950210 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786497534195564-24578-0/minicluster-data/ts-2/wal/instance
I20260812 01:19:51.950430 24578 external_mini_cluster.cc:1428] Setting key a01affb73b24c72adbb322a54ea84f52
I20260812 01:19:51.951332 24578 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260812 01:19:51.953306 7227 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:19:51.952725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48696 (local address 127.24.0.190:43479)
0812 01:19:51.952788 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:51.952790 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:51.952964 (+ 174us) server_negotiation.cc:407] Connection header received
0812 01:19:51.953021 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:51.953022 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:51.953032 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:51.953086 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:51.953236 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48696: BlockingRecv error: recv got EOF from 127.0.0.1:48696 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 01:19:51.953418 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 7264
I20260812 01:19:51.980650 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 7482
I20260812 01:19:51.988042 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 7708
I20260812 01:19:52.000113 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 7101
2026-08-12T01:19:52Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (650 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-12T01:19:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:19:52Z Disabled control of system clock
I20260812 01:19:52.033849 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786497534195564-24578-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786497534195564-24578-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786497534195564-24578-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.190:36611
--webserver_interface=127.24.0.190
--webserver_port=0
--builtin_ntp_servers=127.24.0.148:37649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.0.190:36611
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260812 01:19:52.116036 7939 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:19:52.116251 7939 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:19:52.116273 7939 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:19:52.117337 7939 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:19:52.117370 7939 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:19:52.117383 7939 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:19:52.117395 7939 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:19:52.117408 7939 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:19:52.118798 7939 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:37649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786497534195564-24578-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786497534195564-24578-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.0.190:36611
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.190:36611
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.0.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7939
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786497534195564-24578-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:19:52.118996 7939 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:19:52.119240 7939 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:19:52.123952 7947 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:52.123983 7953 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:19:52.124153 7939 server_base.cc:1034] running on GCE node
W20260812 01:19:52.123992 7945 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:19:52.124462 7939 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:19:52.124765 7939 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:19:52.125911 7939 hybrid_clock.cc:648] HybridClock initialized: now 1786497592125899 us; error 32 us; skew 500 ppm
I20260812 01:19:52.127153 7939 webserver.cc:460] Webserver started at http://127.24.0.190:42729/ using document root <none> and password file <none>
I20260812 01:19:52.127362 7939 fs_manager.cc:359] Metadata directory not provided
I20260812 01:19:52.127403 7939 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:19:52.127493 7939 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:19:52.128582 7939 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786497534195564-24578-0/minicluster-data/master-0/data/instance:
uuid: "a275b5e5d0384d5db9973362b1f14273"
format_stamp: "Formatted at 2026-08-12 01:19:52 on dist-test-slave-t3q3"
server_key: "b4c12dd345978c2c3697e03046f81c2d"
server_key_iv: "bccb3677b6fe6386fa115c493ede8bbf"
server_key_version: "encryptionkey@0"
I20260812 01:19:52.128883 7939 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786497534195564-24578-0/minicluster-data/master-0/wal/instance:
uuid: "a275b5e5d0384d5db9973362b1f14273"
format_stamp: "Formatted at 2026-08-12 01:19:52 on dist-test-slave-t3q3"
server_key: "b4c12dd345978c2c3697e03046f81c2d"
server_key_iv: "bccb3677b6fe6386fa115c493ede8bbf"
server_key_version: "encryptionkey@0"
I20260812 01:19:52.130107 7939 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 01:19:52.130769 7965 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:52.130964 7939 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 01:19:52.131048 7939 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786497534195564-24578-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786497534195564-24578-0/minicluster-data/master-0/wal
uuid: "a275b5e5d0384d5db9973362b1f14273"
format_stamp: "Formatted at 2026-08-12 01:19:52 on dist-test-slave-t3q3"
server_key: "b4c12dd345978c2c3697e03046f81c2d"
server_key_iv: "bccb3677b6fe6386fa115c493ede8bbf"
server_key_version: "encryptionkey@0"
I20260812 01:19:52.131102 7939 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786497534195564-24578-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786497534195564-24578-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786497534195564-24578-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:19:52.134312 7939 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:19:52.134513 7939 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:19:52.134605 7939 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:19:52.138375 7939 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.190:36611
I20260812 01:19:52.138447 8052 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.190:36611 every 8 connection(s)
I20260812 01:19:52.138770 7939 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
I20260812 01:19:52.139454 8053 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:19:52.142223 8053 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a275b5e5d0384d5db9973362b1f14273: Bootstrap starting.
I20260812 01:19:52.142805 8053 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a275b5e5d0384d5db9973362b1f14273: Neither blocks nor log segments found. Creating new log.
I20260812 01:19:52.143121 8053 log.cc:824] T 00000000000000000000000000000000 P a275b5e5d0384d5db9973362b1f14273: Log is configured to *not* fsync() on all Append() calls
I20260812 01:19:52.143815 8053 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a275b5e5d0384d5db9973362b1f14273: No bootstrap required, opened a new log
I20260812 01:19:52.145434 8053 raft_consensus.cc:348] T 00000000000000000000000000000000 P a275b5e5d0384d5db9973362b1f14273 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a275b5e5d0384d5db9973362b1f14273" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 36611 } }
I20260812 01:19:52.145604 8053 raft_consensus.cc:374] T 00000000000000000000000000000000 P a275b5e5d0384d5db9973362b1f14273 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:19:52.145656 8053 raft_consensus.cc:729] T 00000000000000000000000000000000 P a275b5e5d0384d5db9973362b1f14273 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a275b5e5d0384d5db9973362b1f14273, State: Initialized, Role: FOLLOWER
I20260812 01:19:52.145813 8053 consensus_queue.cc:260] T 00000000000000000000000000000000 P a275b5e5d0384d5db9973362b1f14273 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a275b5e5d0384d5db9973362b1f14273" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 36611 } }
I20260812 01:19:52.145895 8053 raft_consensus.cc:388] T 00000000000000000000000000000000 P a275b5e5d0384d5db9973362b1f14273 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:19:52.145941 8053 raft_consensus.cc:482] T 00000000000000000000000000000000 P a275b5e5d0384d5db9973362b1f14273 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:19:52.145988 8053 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a275b5e5d0384d5db9973362b1f14273 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:19:52.146546 8053 raft_consensus.cc:504] T 00000000000000000000000000000000 P a275b5e5d0384d5db9973362b1f14273 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a275b5e5d0384d5db9973362b1f14273" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 36611 } }
I20260812 01:19:52.146690 8053 leader_election.cc:302] T 00000000000000000000000000000000 P a275b5e5d0384d5db9973362b1f14273 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a275b5e5d0384d5db9973362b1f14273; no voters:
I20260812 01:19:52.146903 8053 leader_election.cc:288] T 00000000000000000000000000000000 P a275b5e5d0384d5db9973362b1f14273 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 01:19:52.146952 8059 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a275b5e5d0384d5db9973362b1f14273 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:19:52.147148 8059 raft_consensus.cc:686] T 00000000000000000000000000000000 P a275b5e5d0384d5db9973362b1f14273 [term 1 LEADER]: Becoming Leader. State: Replica: a275b5e5d0384d5db9973362b1f14273, State: Running, Role: LEADER
I20260812 01:19:52.147162 8053 sys_catalog.cc:564] T 00000000000000000000000000000000 P a275b5e5d0384d5db9973362b1f14273 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:19:52.147256 8059 consensus_queue.cc:237] T 00000000000000000000000000000000 P a275b5e5d0384d5db9973362b1f14273 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a275b5e5d0384d5db9973362b1f14273" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 36611 } }
I20260812 01:19:52.147650 8061 sys_catalog.cc:455] T 00000000000000000000000000000000 P a275b5e5d0384d5db9973362b1f14273 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a275b5e5d0384d5db9973362b1f14273. Latest consensus state: current_term: 1 leader_uuid: "a275b5e5d0384d5db9973362b1f14273" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a275b5e5d0384d5db9973362b1f14273" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 36611 } } }
I20260812 01:19:52.147753 8061 sys_catalog.cc:458] T 00000000000000000000000000000000 P a275b5e5d0384d5db9973362b1f14273 [sys.catalog]: This master's current role is: LEADER
I20260812 01:19:52.147929 8060 sys_catalog.cc:455] T 00000000000000000000000000000000 P a275b5e5d0384d5db9973362b1f14273 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a275b5e5d0384d5db9973362b1f14273" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a275b5e5d0384d5db9973362b1f14273" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 36611 } } }
I20260812 01:19:52.148020 8060 sys_catalog.cc:458] T 00000000000000000000000000000000 P a275b5e5d0384d5db9973362b1f14273 [sys.catalog]: This master's current role is: LEADER
I20260812 01:19:52.148383 8068 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:19:52.148384 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 7939
I20260812 01:19:52.148484 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786497534195564-24578-0/minicluster-data/master-0/wal/instance
I20260812 01:19:52.148734 24578 external_mini_cluster.cc:1428] Setting key 9eeb07f96fbda6061cbdca1a6cd23607
I20260812 01:19:52.148873 8068 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:19:52.151083 8089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:52.149417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38116 (local address 127.24.0.190:36611)
0812 01:19:52.149602 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:52.149611 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:52.149641 (+ 30us) server_negotiation.cc:407] Connection header received
0812 01:19:52.149749 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:52.149756 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:52.149796 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:52.149945 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:52.150640 (+ 695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:52.150643 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:52.150643 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:19:52.150647 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:52.150691 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:19:52.150704 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:52.150780 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:52.150834 (+ 54us) server_negotiation.cc:299] Negotiation successful
0812 01:19:52.150888 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":83,"threads_started":1}
I20260812 01:19:52.151364 8068 catalog_manager.cc:1347] Generated new cluster ID: b9a094e714d64a3ea6855095df2f87cd
I20260812 01:19:52.151444 8068 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 01:19:52.158084 8068 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:19:52.158262 8068 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:19:52.162045 8068 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a275b5e5d0384d5db9973362b1f14273: Generated new TSK 0
I20260812 01:19:52.162241 8068 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 01:19:52.166260 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786497534195564-24578-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786497534195564-24578-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786497534195564-24578-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786497534195564-24578-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.0.129:0
--local_ip_for_outbound_sockets=127.24.0.129
--webserver_interface=127.24.0.129
--webserver_port=0
--tserver_master_addrs=127.24.0.190:36611
--builtin_ntp_servers=127.24.0.148:37649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260812 01:19:52.249781 8093 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:19:52.249986 8093 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:19:52.250010 8093 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:19:52.251185 8093 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:19:52.251228 8093 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:19:52.251255 8093 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.0.129
I20260812 01:19:52.252698 8093 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:37649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786497534195564-24578-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786497534195564-24578-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.0.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786497534195564-24578-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.0.129
--webserver_port=0
--tserver_master_addrs=127.24.0.190:36611
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8093
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.0.129
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786497534195564-24578-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:19:52.252955 8093 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:19:52.253311 8093 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:19:52.258419 8101 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:52.258500 8100 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:19:52.258591 8093 server_base.cc:1034] running on GCE node
W20260812 01:19:52.258735 8105 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:19:52.258944 8093 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:19:52.259228 8093 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:19:52.260382 8093 hybrid_clock.cc:648] HybridClock initialized: now 1786497592260372 us; error 35 us; skew 500 ppm
I20260812 01:19:52.261773 8093 webserver.cc:460] Webserver started at http://127.24.0.129:39821/ using document root <none> and password file <none>
I20260812 01:19:52.262035 8093 fs_manager.cc:359] Metadata directory not provided
I20260812 01:19:52.262081 8093 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:19:52.262229 8093 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:19:52.263324 8093 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786497534195564-24578-0/minicluster-data/ts-0/data/instance:
uuid: "13f983f4a25b451aaac14cfc09c9af43"
format_stamp: "Formatted at 2026-08-12 01:19:52 on dist-test-slave-t3q3"
server_key: "b46913ee999b64e8470a23804f5a4022"
server_key_iv: "14503790acef354ed5a2b01249eae170"
server_key_version: "encryptionkey@0"
I20260812 01:19:52.263711 8093 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786497534195564-24578-0/minicluster-data/ts-0/wal/instance:
uuid: "13f983f4a25b451aaac14cfc09c9af43"
format_stamp: "Formatted at 2026-08-12 01:19:52 on dist-test-slave-t3q3"
server_key: "b46913ee999b64e8470a23804f5a4022"
server_key_iv: "14503790acef354ed5a2b01249eae170"
server_key_version: "encryptionkey@0"
I20260812 01:19:52.265187 8093 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 01:19:52.265980 8117 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:52.266151 8093 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 01:19:52.266266 8093 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786497534195564-24578-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786497534195564-24578-0/minicluster-data/ts-0/wal
uuid: "13f983f4a25b451aaac14cfc09c9af43"
format_stamp: "Formatted at 2026-08-12 01:19:52 on dist-test-slave-t3q3"
server_key: "b46913ee999b64e8470a23804f5a4022"
server_key_iv: "14503790acef354ed5a2b01249eae170"
server_key_version: "encryptionkey@0"
I20260812 01:19:52.266341 8093 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786497534195564-24578-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786497534195564-24578-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786497534195564-24578-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:19:52.269820 8093 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:19:52.270103 8093 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:19:52.270253 8093 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:19:52.270465 8093 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:19:52.270795 8093 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:19:52.270829 8093 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:52.270871 8093 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:19:52.270896 8093 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:52.277702 8093 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.129:40225
I20260812 01:19:52.277747 8320 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.129:40225 every 8 connection(s)
I20260812 01:19:52.278079 8093 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786497534195564-24578-0/minicluster-data/ts-0/data/info.pb
I20260812 01:19:52.281561 8089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:52.278845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.129:34005 (local address 127.24.0.190:36611)
0812 01:19:52.278916 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:52.278918 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:52.279198 (+ 280us) server_negotiation.cc:407] Connection header received
0812 01:19:52.279958 (+ 760us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:52.279959 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:52.279965 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:52.280042 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:52.281319 (+ 1277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:52.281321 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:52.281321 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:19:52.281325 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:52.281342 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:19:52.281346 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:52.281407 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:52.281493 (+ 86us) server_negotiation.cc:299] Negotiation successful
0812 01:19:52.281508 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 01:19:52.281765 8325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:52.278823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:36611 (local address 127.24.0.129:34005)
0812 01:19:52.279039 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:52.279084 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:52.279839 (+ 755us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:52.280150 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:52.280154 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:52.280171 (+ 17us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:52.281108 (+ 937us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 01:19:52.281108 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:52.281138 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 01:19:52.281184 (+ 46us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:19:52.281418 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:19:52.281423 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:19:52.281425 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 01:19:52.281487 (+ 62us) client_negotiation.cc:240] Negotiation successful
0812 01:19:52.281539 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":88,"threads_started":1}
I20260812 01:19:52.282044 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 8093
I20260812 01:19:52.282162 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786497534195564-24578-0/minicluster-data/ts-0/wal/instance
I20260812 01:19:52.282338 8322 heartbeater.cc:347] Connected to a master server at 127.24.0.190:36611
I20260812 01:19:52.282421 24578 external_mini_cluster.cc:1428] Setting key 9e4339c4b3b14ec26d2009aa65706a08
I20260812 01:19:52.282442 8322 heartbeater.cc:464] Registering TS with master...
I20260812 01:19:52.282605 8322 heartbeater.cc:511] Master 127.24.0.190:36611 requested a full tablet report, sending...
I20260812 01:19:52.283221 7992 ts_manager.cc:194] Registered new tserver with Master: 13f983f4a25b451aaac14cfc09c9af43 (127.24.0.129:40225)
I20260812 01:19:52.283411 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786497534195564-24578-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786497534195564-24578-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786497534195564-24578-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786497534195564-24578-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.0.130:0
--local_ip_for_outbound_sockets=127.24.0.130
--webserver_interface=127.24.0.130
--webserver_port=0
--tserver_master_addrs=127.24.0.190:36611
--builtin_ntp_servers=127.24.0.148:37649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260812 01:19:52.368628 8326 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:19:52.368999 8326 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:19:52.369031 8326 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:19:52.370304 8326 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:19:52.370347 8326 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:19:52.370373 8326 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.0.130
I20260812 01:19:52.371865 8326 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:37649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786497534195564-24578-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786497534195564-24578-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.0.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786497534195564-24578-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.0.130
--webserver_port=0
--tserver_master_addrs=127.24.0.190:36611
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8326
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.0.130
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786497534195564-24578-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:19:52.372138 8326 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:19:52.372455 8326 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:19:52.377698 8337 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:52.377696 8334 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:52.377835 8333 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:19:52.378153 8326 server_base.cc:1034] running on GCE node
I20260812 01:19:52.378377 8326 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:19:52.378628 8326 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:19:52.379807 8326 hybrid_clock.cc:648] HybridClock initialized: now 1786497592379787 us; error 30 us; skew 500 ppm
I20260812 01:19:52.381167 8326 webserver.cc:460] Webserver started at http://127.24.0.130:42971/ using document root <none> and password file <none>
I20260812 01:19:52.381423 8326 fs_manager.cc:359] Metadata directory not provided
I20260812 01:19:52.381497 8326 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:19:52.381634 8326 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:19:52.382727 8326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786497534195564-24578-0/minicluster-data/ts-1/data/instance:
uuid: "712408087ad84609b13f9bedd80243f9"
format_stamp: "Formatted at 2026-08-12 01:19:52 on dist-test-slave-t3q3"
server_key: "78d062776eed23cf9b202fc65e18744a"
server_key_iv: "f7d028c7f325f62008adf23dba952012"
server_key_version: "encryptionkey@0"
I20260812 01:19:52.383113 8326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786497534195564-24578-0/minicluster-data/ts-1/wal/instance:
uuid: "712408087ad84609b13f9bedd80243f9"
format_stamp: "Formatted at 2026-08-12 01:19:52 on dist-test-slave-t3q3"
server_key: "78d062776eed23cf9b202fc65e18744a"
server_key_iv: "f7d028c7f325f62008adf23dba952012"
server_key_version: "encryptionkey@0"
I20260812 01:19:52.384624 8326 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 01:19:52.385401 8347 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:52.385579 8326 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:19:52.385684 8326 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786497534195564-24578-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786497534195564-24578-0/minicluster-data/ts-1/wal
uuid: "712408087ad84609b13f9bedd80243f9"
format_stamp: "Formatted at 2026-08-12 01:19:52 on dist-test-slave-t3q3"
server_key: "78d062776eed23cf9b202fc65e18744a"
server_key_iv: "f7d028c7f325f62008adf23dba952012"
server_key_version: "encryptionkey@0"
I20260812 01:19:52.385769 8326 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786497534195564-24578-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786497534195564-24578-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786497534195564-24578-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:19:52.389242 8326 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:19:52.389511 8326 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:19:52.389654 8326 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:19:52.389873 8326 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:19:52.390182 8326 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:19:52.390236 8326 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:52.390275 8326 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:19:52.390304 8326 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:52.396216 8326 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.130:40941
I20260812 01:19:52.396255 8547 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.130:40941 every 8 connection(s)
I20260812 01:19:52.396595 8326 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786497534195564-24578-0/minicluster-data/ts-1/data/info.pb
I20260812 01:19:52.399289 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 8326
I20260812 01:19:52.399379 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786497534195564-24578-0/minicluster-data/ts-1/wal/instance
I20260812 01:19:52.399591 24578 external_mini_cluster.cc:1428] Setting key 52fa485d44c709e5b10a05ec74325e60
I20260812 01:19:52.400398 8089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:52.397513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.130:34949 (local address 127.24.0.190:36611)
0812 01:19:52.397637 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:52.397640 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:52.397849 (+ 209us) server_negotiation.cc:407] Connection header received
0812 01:19:52.398453 (+ 604us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:52.398455 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:52.398463 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:52.398514 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:52.400042 (+ 1528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:52.400043 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:52.400043 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:19:52.400047 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:52.400101 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:19:52.400107 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:52.400181 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:52.400302 (+ 121us) server_negotiation.cc:299] Negotiation successful
0812 01:19:52.400320 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260812 01:19:52.400633 8555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:52.397438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:36611 (local address 127.24.0.130:34949)
0812 01:19:52.397711 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:52.397753 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:52.398322 (+ 569us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:52.398602 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:52.398606 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:52.398619 (+ 13us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:52.399852 (+ 1233us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 01:19:52.399853 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:52.399896 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 01:19:52.399933 (+ 37us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:19:52.400197 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:19:52.400202 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:19:52.400206 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 01:19:52.400296 (+ 90us) client_negotiation.cc:240] Negotiation successful
0812 01:19:52.400366 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":94,"threads_started":1}
I20260812 01:19:52.400794 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786497534195564-24578-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786497534195564-24578-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786497534195564-24578-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786497534195564-24578-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.0.131:0
--local_ip_for_outbound_sockets=127.24.0.131
--webserver_interface=127.24.0.131
--webserver_port=0
--tserver_master_addrs=127.24.0.190:36611
--builtin_ntp_servers=127.24.0.148:37649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260812 01:19:52.401075 8552 heartbeater.cc:347] Connected to a master server at 127.24.0.190:36611
I20260812 01:19:52.401201 8552 heartbeater.cc:464] Registering TS with master...
I20260812 01:19:52.401364 8552 heartbeater.cc:511] Master 127.24.0.190:36611 requested a full tablet report, sending...
I20260812 01:19:52.401759 7992 ts_manager.cc:194] Registered new tserver with Master: 712408087ad84609b13f9bedd80243f9 (127.24.0.130:40941)
W20260812 01:19:52.486182 8557 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:19:52.486418 8557 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:19:52.486455 8557 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:19:52.487586 8557 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:19:52.487651 8557 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:19:52.487696 8557 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.0.131
I20260812 01:19:52.489303 8557 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:37649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786497534195564-24578-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786497534195564-24578-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.0.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786497534195564-24578-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.0.131
--webserver_port=0
--tserver_master_addrs=127.24.0.190:36611
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8557
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.0.131
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786497534195564-24578-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:19:52.489567 8557 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:19:52.489871 8557 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:19:52.494978 8577 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:52.494988 8567 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:52.494988 8570 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:19:52.495390 8557 server_base.cc:1034] running on GCE node
I20260812 01:19:52.495635 8557 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:19:52.495887 8557 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:19:52.497021 8557 hybrid_clock.cc:648] HybridClock initialized: now 1786497592497011 us; error 28 us; skew 500 ppm
I20260812 01:19:52.503232 8557 webserver.cc:460] Webserver started at http://127.24.0.131:40943/ using document root <none> and password file <none>
I20260812 01:19:52.503562 8557 fs_manager.cc:359] Metadata directory not provided
I20260812 01:19:52.503649 8557 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:19:52.503818 8557 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:19:52.505014 8557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786497534195564-24578-0/minicluster-data/ts-2/data/instance:
uuid: "2a80718de93c460b9b493240273a0564"
format_stamp: "Formatted at 2026-08-12 01:19:52 on dist-test-slave-t3q3"
server_key: "e996cd94a6169bc45a8448a46e308527"
server_key_iv: "a094fcb3ad3418a9b105dab5394ce924"
server_key_version: "encryptionkey@0"
I20260812 01:19:52.505395 8557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786497534195564-24578-0/minicluster-data/ts-2/wal/instance:
uuid: "2a80718de93c460b9b493240273a0564"
format_stamp: "Formatted at 2026-08-12 01:19:52 on dist-test-slave-t3q3"
server_key: "e996cd94a6169bc45a8448a46e308527"
server_key_iv: "a094fcb3ad3418a9b105dab5394ce924"
server_key_version: "encryptionkey@0"
I20260812 01:19:52.506822 8557 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 01:19:52.507616 8586 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:52.507833 8557 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 01:19:52.507937 8557 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786497534195564-24578-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786497534195564-24578-0/minicluster-data/ts-2/wal
uuid: "2a80718de93c460b9b493240273a0564"
format_stamp: "Formatted at 2026-08-12 01:19:52 on dist-test-slave-t3q3"
server_key: "e996cd94a6169bc45a8448a46e308527"
server_key_iv: "a094fcb3ad3418a9b105dab5394ce924"
server_key_version: "encryptionkey@0"
I20260812 01:19:52.508024 8557 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786497534195564-24578-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786497534195564-24578-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786497534195564-24578-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:19:52.511461 8557 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:19:52.511747 8557 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:19:52.511894 8557 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:19:52.512156 8557 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:19:52.512486 8557 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:19:52.512547 8557 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:52.512586 8557 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:19:52.512612 8557 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:52.518882 8557 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.131:45325
I20260812 01:19:52.518929 8798 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.131:45325 every 8 connection(s)
I20260812 01:19:52.519269 8557 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786497534195564-24578-0/minicluster-data/ts-2/data/info.pb
I20260812 01:19:52.522481 8089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:52.520102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.131:38343 (local address 127.24.0.190:36611)
0812 01:19:52.520165 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:52.520172 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:52.520392 (+ 220us) server_negotiation.cc:407] Connection header received
0812 01:19:52.520937 (+ 545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:52.520939 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:52.520945 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:52.521014 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:52.522179 (+ 1165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:52.522180 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:52.522180 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:19:52.522184 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:52.522200 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:19:52.522204 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:52.522259 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:52.522409 (+ 150us) server_negotiation.cc:299] Negotiation successful
0812 01:19:52.522426 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260812 01:19:52.522560 8807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:52.520036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:36611 (local address 127.24.0.131:38343)
0812 01:19:52.520238 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:52.520279 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:52.520836 (+ 557us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:52.521095 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:52.521098 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:52.521107 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:52.522019 (+ 912us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 01:19:52.522020 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:52.522054 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 01:19:52.522083 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:19:52.522272 (+ 189us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:19:52.522276 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:19:52.522279 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 01:19:52.522338 (+ 59us) client_negotiation.cc:240] Negotiation successful
0812 01:19:52.522388 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":103,"threads_started":1}
I20260812 01:19:52.523132 8804 heartbeater.cc:347] Connected to a master server at 127.24.0.190:36611
I20260812 01:19:52.523252 8804 heartbeater.cc:464] Registering TS with master...
I20260812 01:19:52.523422 8804 heartbeater.cc:511] Master 127.24.0.190:36611 requested a full tablet report, sending...
I20260812 01:19:52.523845 7992 ts_manager.cc:194] Registered new tserver with Master: 2a80718de93c460b9b493240273a0564 (127.24.0.131:45325)
I20260812 01:19:52.526978 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 8557
I20260812 01:19:52.527068 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786497534195564-24578-0/minicluster-data/ts-2/wal/instance
I20260812 01:19:52.527303 24578 external_mini_cluster.cc:1428] Setting key c3bce7be8c3cb1ee70ae628e441aaf0d
I20260812 01:19:52.528373 24578 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260812 01:19:52.530355 8089 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:19:52.529774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38126 (local address 127.24.0.190:36611)
0812 01:19:52.529859 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:52.529861 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:52.529959 (+ 98us) server_negotiation.cc:407] Connection header received
0812 01:19:52.530081 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:52.530081 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:52.530086 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:52.530152 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:52.530279 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38126: BlockingRecv error: recv got EOF from 127.0.0.1:38126 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 01:19:52.530457 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 8093
I20260812 01:19:52.542824 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 8326
I20260812 01:19:52.552861 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 8557
I20260812 01:19:52.569195 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 7939
2026-08-12T01:19:52Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (569 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786497534195564-24578-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 01:19:52 dist-test-slave-t3q3 krb5kdc[8819](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 01:19:52 dist-test-slave-t3q3 krb5kdc[8819](info): set up 2 sockets
Aug 12 01:19:52 dist-test-slave-t3q3 krb5kdc[8819](info): commencing operation
krb5kdc: starting...
W20260812 01:19:53.858994 24578 mini_kdc.cc:121] Time spent starting KDC: real 1.264s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786497534195564-24578-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786497534195564-24578-0/krb5kdc/test-user.keytab.
Aug 12 01:19:53 dist-test-slave-t3q3 krb5kdc[8819](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497593, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T01:19:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:19:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.0.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.0.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.0.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab.
Entry for principal kudu/127.24.0.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab.
I20260812 01:19:53.994534 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786497534195564-24578-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786497534195564-24578-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786497534195564-24578-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.190:35681
--webserver_interface=127.24.0.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab
--principal=kudu/127.24.0.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:33049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.0.190:35681
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:19:54.080897 8843 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:19:54.081107 8843 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:19:54.081130 8843 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:19:54.082216 8843 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:19:54.082252 8843 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:19:54.082266 8843 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:19:54.082279 8843 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:19:54.082291 8843 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:19:54.083719 8843 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:33049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786497534195564-24578-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786497534195564-24578-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.0.190:35681
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab
--principal=kudu/127.24.0.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.190:35681
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.0.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8843
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786497534195564-24578-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:19:54.083923 8843 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:19:54.084142 8843 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:19:54.088967 8852 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:54.088976 8857 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:54.088976 8851 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:19:54.089208 8843 server_base.cc:1034] running on GCE node
I20260812 01:19:54.089479 8843 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:19:54.089723 8843 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:19:54.090922 8843 hybrid_clock.cc:648] HybridClock initialized: now 1786497594090905 us; error 37 us; skew 500 ppm
Aug 12 01:19:54 dist-test-slave-t3q3 krb5kdc[8819](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497594, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:19:54.093698 8843 init.cc:377] Logged in from keytab as kudu/127.24.0.190@KRBTEST.COM (short username kudu)
I20260812 01:19:54.094193 8843 webserver.cc:460] Webserver started at http://127.24.0.190:37331/ using document root <none> and password file <none>
I20260812 01:19:54.094446 8843 fs_manager.cc:359] Metadata directory not provided
I20260812 01:19:54.094492 8843 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:19:54.094648 8843 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:19:54.095709 8843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786497534195564-24578-0/minicluster-data/master-0/data/instance:
uuid: "8d7424c39e3b4743a2e09f70dbfb5b2a"
format_stamp: "Formatted at 2026-08-12 01:19:54 on dist-test-slave-t3q3"
server_key: "bf696c56cffbd49942a55d88d4d50c3a"
server_key_iv: "fae6486b4f0d0237584c9d35af3be4db"
server_key_version: "encryptionkey@0"
I20260812 01:19:54.096149 8843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786497534195564-24578-0/minicluster-data/master-0/wal/instance:
uuid: "8d7424c39e3b4743a2e09f70dbfb5b2a"
format_stamp: "Formatted at 2026-08-12 01:19:54 on dist-test-slave-t3q3"
server_key: "bf696c56cffbd49942a55d88d4d50c3a"
server_key_iv: "fae6486b4f0d0237584c9d35af3be4db"
server_key_version: "encryptionkey@0"
I20260812 01:19:54.097517 8843 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 01:19:54.098312 8868 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:54.098618 8843 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 01:19:54.098727 8843 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786497534195564-24578-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786497534195564-24578-0/minicluster-data/master-0/wal
uuid: "8d7424c39e3b4743a2e09f70dbfb5b2a"
format_stamp: "Formatted at 2026-08-12 01:19:54 on dist-test-slave-t3q3"
server_key: "bf696c56cffbd49942a55d88d4d50c3a"
server_key_iv: "fae6486b4f0d0237584c9d35af3be4db"
server_key_version: "encryptionkey@0"
I20260812 01:19:54.098804 8843 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786497534195564-24578-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786497534195564-24578-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786497534195564-24578-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:19:54.114209 8843 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:19:54.116223 8843 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:19:54.116400 8843 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:19:54.120666 8843 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.190:35681
I20260812 01:19:54.120718 8971 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.190:35681 every 8 connection(s)
I20260812 01:19:54.121124 8843 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
I20260812 01:19:54.122171 8972 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:19:54.124557 8972 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d7424c39e3b4743a2e09f70dbfb5b2a: Bootstrap starting.
I20260812 01:19:54.125123 8972 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8d7424c39e3b4743a2e09f70dbfb5b2a: Neither blocks nor log segments found. Creating new log.
I20260812 01:19:54.125391 8972 log.cc:824] T 00000000000000000000000000000000 P 8d7424c39e3b4743a2e09f70dbfb5b2a: Log is configured to *not* fsync() on all Append() calls
I20260812 01:19:54.126111 8972 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d7424c39e3b4743a2e09f70dbfb5b2a: No bootstrap required, opened a new log
I20260812 01:19:54.127681 8972 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8d7424c39e3b4743a2e09f70dbfb5b2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d7424c39e3b4743a2e09f70dbfb5b2a" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 35681 } }
I20260812 01:19:54.127821 8972 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8d7424c39e3b4743a2e09f70dbfb5b2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:19:54.127844 8972 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8d7424c39e3b4743a2e09f70dbfb5b2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d7424c39e3b4743a2e09f70dbfb5b2a, State: Initialized, Role: FOLLOWER
I20260812 01:19:54.127981 8972 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8d7424c39e3b4743a2e09f70dbfb5b2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d7424c39e3b4743a2e09f70dbfb5b2a" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 35681 } }
I20260812 01:19:54.128119 8972 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8d7424c39e3b4743a2e09f70dbfb5b2a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:19:54.128167 8972 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8d7424c39e3b4743a2e09f70dbfb5b2a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:19:54.128201 8972 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8d7424c39e3b4743a2e09f70dbfb5b2a [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:19:54.129016 8972 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8d7424c39e3b4743a2e09f70dbfb5b2a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d7424c39e3b4743a2e09f70dbfb5b2a" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 35681 } }
I20260812 01:19:54.129108 8972 leader_election.cc:302] T 00000000000000000000000000000000 P 8d7424c39e3b4743a2e09f70dbfb5b2a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8d7424c39e3b4743a2e09f70dbfb5b2a; no voters:
I20260812 01:19:54.129228 8972 leader_election.cc:288] T 00000000000000000000000000000000 P 8d7424c39e3b4743a2e09f70dbfb5b2a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 01:19:54.129300 8975 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8d7424c39e3b4743a2e09f70dbfb5b2a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:19:54.129485 8972 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8d7424c39e3b4743a2e09f70dbfb5b2a [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:19:54.129511 8975 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8d7424c39e3b4743a2e09f70dbfb5b2a [term 1 LEADER]: Becoming Leader. State: Replica: 8d7424c39e3b4743a2e09f70dbfb5b2a, State: Running, Role: LEADER
I20260812 01:19:54.129617 8975 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8d7424c39e3b4743a2e09f70dbfb5b2a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d7424c39e3b4743a2e09f70dbfb5b2a" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 35681 } }
I20260812 01:19:54.130038 8977 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d7424c39e3b4743a2e09f70dbfb5b2a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8d7424c39e3b4743a2e09f70dbfb5b2a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d7424c39e3b4743a2e09f70dbfb5b2a" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 35681 } } }
I20260812 01:19:54.130142 8977 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d7424c39e3b4743a2e09f70dbfb5b2a [sys.catalog]: This master's current role is: LEADER
I20260812 01:19:54.130363 8994 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:19:54.130561 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 8843
I20260812 01:19:54.130681 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786497534195564-24578-0/minicluster-data/master-0/wal/instance
I20260812 01:19:54.130930 24578 external_mini_cluster.cc:1428] Setting key 9543467ce5d1feb3688f77a2feff2610
I20260812 01:19:54.130959 8994 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:19:54.131062 8979 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d7424c39e3b4743a2e09f70dbfb5b2a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8d7424c39e3b4743a2e09f70dbfb5b2a. Latest consensus state: current_term: 1 leader_uuid: "8d7424c39e3b4743a2e09f70dbfb5b2a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d7424c39e3b4743a2e09f70dbfb5b2a" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 35681 } } }
I20260812 01:19:54.131151 8979 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d7424c39e3b4743a2e09f70dbfb5b2a [sys.catalog]: This master's current role is: LEADER
I20260812 01:19:54.134002 8994 catalog_manager.cc:1347] Generated new cluster ID: d6c673e62cc24f14a3d538fec1e3b79a
I20260812 01:19:54.134075 8994 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 01:19:54 dist-test-slave-t3q3 krb5kdc[8819](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497593, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
I20260812 01:19:54.142575 9012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:54.131483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33538 (local address 127.24.0.190:35681)
0812 01:19:54.131687 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:54.131697 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:54.131736 (+ 39us) server_negotiation.cc:407] Connection header received
0812 01:19:54.131853 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:54.131860 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:54.131922 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:54.132108 (+ 186us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:54.132926 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:54.133838 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:54.134888 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:54.135535 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:54.135664 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:54.138583 (+ 2919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:54.138589 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:54.138590 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:54.138606 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:54.140875 (+ 2269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:54.141267 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:54.141270 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:54.141271 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:54.141403 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:54.141625 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:54.141626 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:54.141626 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:54.141953 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:54.142118 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:54.142195 (+ 77us) server_negotiation.cc:299] Negotiation successful
0812 01:19:54.142337 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":89,"threads_started":1}
I20260812 01:19:54.144161 8994 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:19:54.144688 8994 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:19:54.149320 8994 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8d7424c39e3b4743a2e09f70dbfb5b2a: Generated new TSK 0
I20260812 01:19:54.149513 8994 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.0.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.0.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.0.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.129.keytab.
Entry for principal kudu/127.24.0.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.129.keytab.
I20260812 01:19:54.186868 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786497534195564-24578-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786497534195564-24578-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786497534195564-24578-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786497534195564-24578-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.0.129:0
--local_ip_for_outbound_sockets=127.24.0.129
--webserver_interface=127.24.0.129
--webserver_port=0
--tserver_master_addrs=127.24.0.190:35681
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.129.keytab
--principal=kudu/127.24.0.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:33049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:19:54.284582 9018 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:19:54.284837 9018 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:19:54.284883 9018 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:19:54.286489 9018 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:19:54.286574 9018 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:19:54.286623 9018 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.0.129
I20260812 01:19:54.288365 9018 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:33049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786497534195564-24578-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786497534195564-24578-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.129.keytab
--principal=kudu/127.24.0.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.0.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786497534195564-24578-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.0.129
--webserver_port=0
--tserver_master_addrs=127.24.0.190:35681
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9018
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.0.129
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786497534195564-24578-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:19:54.288672 9018 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:19:54.288925 9018 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:19:54.294224 9029 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:54.294262 9033 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:54.294281 9028 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:19:54.294530 9018 server_base.cc:1034] running on GCE node
I20260812 01:19:54.294734 9018 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:19:54.294979 9018 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:19:54.296162 9018 hybrid_clock.cc:648] HybridClock initialized: now 1786497594296136 us; error 39 us; skew 500 ppm
Aug 12 01:19:54 dist-test-slave-t3q3 krb5kdc[8819](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497594, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:19:54.298720 9018 init.cc:377] Logged in from keytab as kudu/127.24.0.129@KRBTEST.COM (short username kudu)
I20260812 01:19:54.299175 9018 webserver.cc:460] Webserver started at http://127.24.0.129:45443/ using document root <none> and password file <none>
I20260812 01:19:54.299410 9018 fs_manager.cc:359] Metadata directory not provided
I20260812 01:19:54.299481 9018 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:19:54.299652 9018 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:19:54.300805 9018 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786497534195564-24578-0/minicluster-data/ts-0/data/instance:
uuid: "19b7bd8107cd45d2a440defafeb59ee2"
format_stamp: "Formatted at 2026-08-12 01:19:54 on dist-test-slave-t3q3"
server_key: "d8afc3b27304ad9d47e61035770e72c7"
server_key_iv: "de1d9b489a163fd532ecaaf658501f96"
server_key_version: "encryptionkey@0"
I20260812 01:19:54.301184 9018 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786497534195564-24578-0/minicluster-data/ts-0/wal/instance:
uuid: "19b7bd8107cd45d2a440defafeb59ee2"
format_stamp: "Formatted at 2026-08-12 01:19:54 on dist-test-slave-t3q3"
server_key: "d8afc3b27304ad9d47e61035770e72c7"
server_key_iv: "de1d9b489a163fd532ecaaf658501f96"
server_key_version: "encryptionkey@0"
I20260812 01:19:54.302678 9018 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 01:19:54.303480 9043 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:54.303679 9018 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:19:54.303795 9018 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786497534195564-24578-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786497534195564-24578-0/minicluster-data/ts-0/wal
uuid: "19b7bd8107cd45d2a440defafeb59ee2"
format_stamp: "Formatted at 2026-08-12 01:19:54 on dist-test-slave-t3q3"
server_key: "d8afc3b27304ad9d47e61035770e72c7"
server_key_iv: "de1d9b489a163fd532ecaaf658501f96"
server_key_version: "encryptionkey@0"
I20260812 01:19:54.303890 9018 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786497534195564-24578-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786497534195564-24578-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786497534195564-24578-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:19:54.317114 9018 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:19:54.318946 9018 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:19:54.319118 9018 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:19:54.319355 9018 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:19:54.319682 9018 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:19:54.319741 9018 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:54.319780 9018 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:19:54.319808 9018 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:54.325788 9018 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.129:44909
I20260812 01:19:54.325831 9238 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.129:44909 every 8 connection(s)
I20260812 01:19:54.326188 9018 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786497534195564-24578-0/minicluster-data/ts-0/data/info.pb
Aug 12 01:19:54 dist-test-slave-t3q3 krb5kdc[8819](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497594, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.129@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
I20260812 01:19:54.333304 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 9018
I20260812 01:19:54.333423 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786497534195564-24578-0/minicluster-data/ts-0/wal/instance
I20260812 01:19:54.333633 24578 external_mini_cluster.cc:1428] Setting key f285e998592e87b76dcc3a1f5d2458ed
I20260812 01:19:54.336660 9012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:54.326997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.129:59359 (local address 127.24.0.190:35681)
0812 01:19:54.327050 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:54.327052 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:54.327266 (+ 214us) server_negotiation.cc:407] Connection header received
0812 01:19:54.327727 (+ 461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:54.327728 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:54.327742 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:54.327805 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:54.328877 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:54.329429 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:54.330507 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:54.330771 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:54.330829 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:54.333175 (+ 2346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:54.333179 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:54.333179 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:54.333191 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:54.335073 (+ 1882us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:54.335482 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:54.335483 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:54.335484 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:54.335557 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:54.335798 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:54.335800 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:54.335800 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:54.335954 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:54.336023 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:54.336511 (+ 488us) server_negotiation.cc:299] Negotiation successful
0812 01:19:54.336598 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 01:19:54.337028 9242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:54.326939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:35681 (local address 127.24.0.129:59359)
0812 01:19:54.327115 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:54.327157 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:54.327640 (+ 483us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:54.327895 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:54.327898 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:54.328572 (+ 674us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:54.328762 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:54.328767 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:54.329552 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:54.329553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:54.330423 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:54.330426 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:54.330851 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:54.330852 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:54.330955 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:54.331507 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:54.331516 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:54.332974 (+ 1458us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:19:54.335164 (+ 2190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:54.335172 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:54.335173 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:54.335403 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:54.335627 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:54.335628 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:54.335628 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:54.335731 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:54.336144 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:19:54.336152 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:19:54.336339 (+ 187us) client_negotiation.cc:769] Sending connection context
0812 01:19:54.336479 (+ 140us) client_negotiation.cc:240] Negotiation successful
0812 01:19:54.336668 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":98,"threads_started":1}
I20260812 01:19:54.337630 9240 heartbeater.cc:347] Connected to a master server at 127.24.0.190:35681
I20260812 01:19:54.337738 9240 heartbeater.cc:464] Registering TS with master...
I20260812 01:19:54.337934 9240 heartbeater.cc:511] Master 127.24.0.190:35681 requested a full tablet report, sending...
I20260812 01:19:54.338452 8896 ts_manager.cc:194] Registered new tserver with Master: 19b7bd8107cd45d2a440defafeb59ee2 (127.24.0.129:44909)
I20260812 01:19:54.339138 8896 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.0.129@KRBTEST.COM'} at 127.24.0.129:59359
WARNING: no policy specified for kudu/127.24.0.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.0.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.0.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.130.keytab.
Entry for principal kudu/127.24.0.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.130.keytab.
I20260812 01:19:54.362633 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786497534195564-24578-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786497534195564-24578-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786497534195564-24578-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786497534195564-24578-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.0.130:0
--local_ip_for_outbound_sockets=127.24.0.130
--webserver_interface=127.24.0.130
--webserver_port=0
--tserver_master_addrs=127.24.0.190:35681
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.130.keytab
--principal=kudu/127.24.0.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:33049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:19:54.446870 9247 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:19:54.447096 9247 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:19:54.447119 9247 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:19:54.448285 9247 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:19:54.448328 9247 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:19:54.448354 9247 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.0.130
I20260812 01:19:54.449759 9247 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:33049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786497534195564-24578-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786497534195564-24578-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.130.keytab
--principal=kudu/127.24.0.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.0.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786497534195564-24578-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.0.130
--webserver_port=0
--tserver_master_addrs=127.24.0.190:35681
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9247
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.0.130
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786497534195564-24578-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:19:54.449946 9247 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:19:54.450140 9247 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:19:54.455464 9260 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:19:54.455478 9247 server_base.cc:1034] running on GCE node
W20260812 01:19:54.455439 9262 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:54.455447 9255 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:19:54.455904 9247 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:19:54.456221 9247 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:19:54.457401 9247 hybrid_clock.cc:648] HybridClock initialized: now 1786497594457395 us; error 37 us; skew 500 ppm
Aug 12 01:19:54 dist-test-slave-t3q3 krb5kdc[8819](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497594, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:19:54.460028 9247 init.cc:377] Logged in from keytab as kudu/127.24.0.130@KRBTEST.COM (short username kudu)
I20260812 01:19:54.460546 9247 webserver.cc:460] Webserver started at http://127.24.0.130:35671/ using document root <none> and password file <none>
I20260812 01:19:54.460741 9247 fs_manager.cc:359] Metadata directory not provided
I20260812 01:19:54.460783 9247 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:19:54.460876 9247 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:19:54.461889 9247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786497534195564-24578-0/minicluster-data/ts-1/data/instance:
uuid: "76786ee43cc0428b91f4e96437817565"
format_stamp: "Formatted at 2026-08-12 01:19:54 on dist-test-slave-t3q3"
server_key: "0d66de81f10b2c8b80bfccc0efbcf9a0"
server_key_iv: "8e4c02850d8f93b829ad77f2d5642719"
server_key_version: "encryptionkey@0"
I20260812 01:19:54.462199 9247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786497534195564-24578-0/minicluster-data/ts-1/wal/instance:
uuid: "76786ee43cc0428b91f4e96437817565"
format_stamp: "Formatted at 2026-08-12 01:19:54 on dist-test-slave-t3q3"
server_key: "0d66de81f10b2c8b80bfccc0efbcf9a0"
server_key_iv: "8e4c02850d8f93b829ad77f2d5642719"
server_key_version: "encryptionkey@0"
I20260812 01:19:54.463536 9247 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 01:19:54.464340 9276 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:54.464519 9247 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:19:54.464622 9247 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786497534195564-24578-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786497534195564-24578-0/minicluster-data/ts-1/wal
uuid: "76786ee43cc0428b91f4e96437817565"
format_stamp: "Formatted at 2026-08-12 01:19:54 on dist-test-slave-t3q3"
server_key: "0d66de81f10b2c8b80bfccc0efbcf9a0"
server_key_iv: "8e4c02850d8f93b829ad77f2d5642719"
server_key_version: "encryptionkey@0"
I20260812 01:19:54.464679 9247 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786497534195564-24578-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786497534195564-24578-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786497534195564-24578-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:19:54.482774 9247 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:19:54.485260 9247 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:19:54.485386 9247 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:19:54.485638 9247 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:19:54.485962 9247 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:19:54.485997 9247 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:54.486020 9247 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:19:54.486033 9247 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:54.491935 9247 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.130:44481
I20260812 01:19:54.492014 9470 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.130:44481 every 8 connection(s)
I20260812 01:19:54.492360 9247 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786497534195564-24578-0/minicluster-data/ts-1/data/info.pb
I20260812 01:19:54.498157 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 9247
I20260812 01:19:54.498281 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786497534195564-24578-0/minicluster-data/ts-1/wal/instance
I20260812 01:19:54.498520 24578 external_mini_cluster.cc:1428] Setting key 274cf4abdb2106a1aa95e6eac596d38a
Aug 12 01:19:54 dist-test-slave-t3q3 krb5kdc[8819](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497594, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.130@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
I20260812 01:19:54.502432 9012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:54.493239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.130:51505 (local address 127.24.0.190:35681)
0812 01:19:54.493291 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:54.493293 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:54.493348 (+ 55us) server_negotiation.cc:407] Connection header received
0812 01:19:54.493840 (+ 492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:54.493841 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:54.493852 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:54.493910 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:54.494956 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:54.495503 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:54.496619 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:54.496815 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:54.496893 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:54.499397 (+ 2504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:54.499399 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:54.499399 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:54.499408 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:54.500957 (+ 1549us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:54.501379 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:54.501380 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:54.501380 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:54.501441 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:54.501717 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:54.501718 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:54.501718 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:54.501836 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:54.501917 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:54.502227 (+ 310us) server_negotiation.cc:299] Negotiation successful
0812 01:19:54.502317 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 01:19:54.502630 9476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:54.493070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:35681 (local address 127.24.0.130:51505)
0812 01:19:54.493233 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:54.493276 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:54.493754 (+ 478us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:54.493978 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:54.493981 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:54.494586 (+ 605us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:54.494834 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:54.494839 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:54.495601 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:54.495602 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:54.496529 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:54.496531 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:54.496924 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:54.496925 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:54.497035 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:54.497538 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:54.497546 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:54.499301 (+ 1755us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:19:54.501039 (+ 1738us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:54.501047 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:54.501049 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:54.501304 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:54.501511 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:54.501512 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:54.501512 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:54.501638 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:54.501945 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:19:54.501951 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:19:54.502090 (+ 139us) client_negotiation.cc:769] Sending connection context
0812 01:19:54.502204 (+ 114us) client_negotiation.cc:240] Negotiation successful
0812 01:19:54.502370 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":91,"threads_started":1}
I20260812 01:19:54.503098 9472 heartbeater.cc:347] Connected to a master server at 127.24.0.190:35681
I20260812 01:19:54.503225 9472 heartbeater.cc:464] Registering TS with master...
I20260812 01:19:54.503409 9472 heartbeater.cc:511] Master 127.24.0.190:35681 requested a full tablet report, sending...
I20260812 01:19:54.503825 8896 ts_manager.cc:194] Registered new tserver with Master: 76786ee43cc0428b91f4e96437817565 (127.24.0.130:44481)
I20260812 01:19:54.504283 8896 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.0.130@KRBTEST.COM'} at 127.24.0.130:51505
WARNING: no policy specified for kudu/127.24.0.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.0.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.0.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.131.keytab.
Entry for principal kudu/127.24.0.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.131.keytab.
I20260812 01:19:54.526587 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786497534195564-24578-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786497534195564-24578-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786497534195564-24578-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786497534195564-24578-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.0.131:0
--local_ip_for_outbound_sockets=127.24.0.131
--webserver_interface=127.24.0.131
--webserver_port=0
--tserver_master_addrs=127.24.0.190:35681
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.131.keytab
--principal=kudu/127.24.0.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:33049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:19:54.610534 9485 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:19:54.610785 9485 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:19:54.610826 9485 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:19:54.611974 9485 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:19:54.612039 9485 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:19:54.612105 9485 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.0.131
I20260812 01:19:54.613600 9485 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:33049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786497534195564-24578-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786497534195564-24578-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.131.keytab
--principal=kudu/127.24.0.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.0.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786497534195564-24578-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.0.131
--webserver_port=0
--tserver_master_addrs=127.24.0.190:35681
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9485
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.0.131
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786497534195564-24578-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:19:54.613875 9485 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:19:54.614108 9485 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:19:54.619561 9495 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:54.619603 9492 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:19:54.619575 9493 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:19:54.619837 9485 server_base.cc:1034] running on GCE node
I20260812 01:19:54.620182 9485 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:19:54.620461 9485 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:19:54.621657 9485 hybrid_clock.cc:648] HybridClock initialized: now 1786497594621637 us; error 32 us; skew 500 ppm
Aug 12 01:19:54 dist-test-slave-t3q3 krb5kdc[8819](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497594, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:19:54.624696 9485 init.cc:377] Logged in from keytab as kudu/127.24.0.131@KRBTEST.COM (short username kudu)
I20260812 01:19:54.625201 9485 webserver.cc:460] Webserver started at http://127.24.0.131:45611/ using document root <none> and password file <none>
I20260812 01:19:54.625474 9485 fs_manager.cc:359] Metadata directory not provided
I20260812 01:19:54.625551 9485 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:19:54.625716 9485 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:19:54.626864 9485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786497534195564-24578-0/minicluster-data/ts-2/data/instance:
uuid: "3fb04fb44ec0473eb4cda10636008e23"
format_stamp: "Formatted at 2026-08-12 01:19:54 on dist-test-slave-t3q3"
server_key: "93f79cc7dc0c5e52000531df337b6fdc"
server_key_iv: "f2fbd04085bc3e0ac9e85fba68af74d8"
server_key_version: "encryptionkey@0"
I20260812 01:19:54.627293 9485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786497534195564-24578-0/minicluster-data/ts-2/wal/instance:
uuid: "3fb04fb44ec0473eb4cda10636008e23"
format_stamp: "Formatted at 2026-08-12 01:19:54 on dist-test-slave-t3q3"
server_key: "93f79cc7dc0c5e52000531df337b6fdc"
server_key_iv: "f2fbd04085bc3e0ac9e85fba68af74d8"
server_key_version: "encryptionkey@0"
I20260812 01:19:54.628871 9485 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 01:19:54.629828 9507 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:54.630100 9485 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 01:19:54.630209 9485 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786497534195564-24578-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786497534195564-24578-0/minicluster-data/ts-2/wal
uuid: "3fb04fb44ec0473eb4cda10636008e23"
format_stamp: "Formatted at 2026-08-12 01:19:54 on dist-test-slave-t3q3"
server_key: "93f79cc7dc0c5e52000531df337b6fdc"
server_key_iv: "f2fbd04085bc3e0ac9e85fba68af74d8"
server_key_version: "encryptionkey@0"
I20260812 01:19:54.630319 9485 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786497534195564-24578-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786497534195564-24578-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786497534195564-24578-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:19:54.650729 9485 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:19:54.653533 9485 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:19:54.653723 9485 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:19:54.654003 9485 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:19:54.654368 9485 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:19:54.654429 9485 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:54.654479 9485 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:19:54.654506 9485 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:19:54.660979 9485 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.131:34925
I20260812 01:19:54.661037 9731 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.131:34925 every 8 connection(s)
I20260812 01:19:54.661418 9485 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786497534195564-24578-0/minicluster-data/ts-2/data/info.pb
I20260812 01:19:54.662449 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 9485
I20260812 01:19:54.662567 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786497534195564-24578-0/minicluster-data/ts-2/wal/instance
I20260812 01:19:54.662802 24578 external_mini_cluster.cc:1428] Setting key b9ddb6edf62674782a2f1bf5195145f6
Aug 12 01:19:54 dist-test-slave-t3q3 krb5kdc[8819](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497594, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.131@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
I20260812 01:19:54.671851 9012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:54.662229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.131:40891 (local address 127.24.0.190:35681)
0812 01:19:54.662316 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:54.662318 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:54.662593 (+ 275us) server_negotiation.cc:407] Connection header received
0812 01:19:54.663391 (+ 798us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:54.663391 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:54.663420 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:54.663480 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:54.664799 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:54.665290 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:54.666329 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:54.666511 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:54.666567 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:54.669099 (+ 2532us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:54.669101 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:54.669101 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:54.669108 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:54.670449 (+ 1341us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:54.670881 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:54.670882 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:54.670882 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:54.670942 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:54.671244 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:54.671245 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:54.671245 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:54.671366 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:54.671426 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:54.671702 (+ 276us) server_negotiation.cc:299] Negotiation successful
0812 01:19:54.671774 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 01:19:54.672020 9734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:54.662216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:35681 (local address 127.24.0.131:40891)
0812 01:19:54.662428 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:54.662482 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:54.663250 (+ 768us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:54.663698 (+ 448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:54.663703 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:54.664496 (+ 793us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:19:54.664714 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:54.664719 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:54.665378 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:54.665379 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:54.666247 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:54.666250 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:54.666590 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:54.666591 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:54.666696 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:54.667220 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:19:54.667229 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:19:54.668999 (+ 1770us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:19:54.670548 (+ 1549us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:54.670555 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:54.670557 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:54.670787 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:54.671036 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:19:54.671037 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:19:54.671038 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:19:54.671155 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:19:54.671434 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:19:54.671439 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:19:54.671565 (+ 126us) client_negotiation.cc:769] Sending connection context
0812 01:19:54.671656 (+ 91us) client_negotiation.cc:240] Negotiation successful
0812 01:19:54.671786 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":89,"threads_started":1}
I20260812 01:19:54.672566 9732 heartbeater.cc:347] Connected to a master server at 127.24.0.190:35681
I20260812 01:19:54.672676 9732 heartbeater.cc:464] Registering TS with master...
I20260812 01:19:54.672879 9732 heartbeater.cc:511] Master 127.24.0.190:35681 requested a full tablet report, sending...
I20260812 01:19:54.673282 8896 ts_manager.cc:194] Registered new tserver with Master: 3fb04fb44ec0473eb4cda10636008e23 (127.24.0.131:34925)
I20260812 01:19:54.673695 8896 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.0.131@KRBTEST.COM'} at 127.24.0.131:40891
I20260812 01:19:54.674813 24578 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 01:19:54.682237 9012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:54.676275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33552 (local address 127.24.0.190:35681)
0812 01:19:54.676321 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:54.676323 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:54.676467 (+ 144us) server_negotiation.cc:407] Connection header received
0812 01:19:54.676504 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:54.676505 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:54.676514 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:54.676547 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:19:54.677180 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:54.677662 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:54.678494 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:54.678678 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:54.678741 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:54.679940 (+ 1199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:19:54.679941 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:19:54.679941 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:19:54.679947 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:19:54.681261 (+ 1314us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:54.681581 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:54.681582 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:54.681582 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:54.681644 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:19:54.681780 (+ 136us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:19:54.681781 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:19:54.681781 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:19:54.681914 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:19:54.682016 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:54.682087 (+ 71us) server_negotiation.cc:299] Negotiation successful
0812 01:19:54.682151 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 01:19:54.683921 8896 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33552:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 01:19:54.685591 8896 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 01:19:54.692193 9746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:54.687927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.129:44909 (local address 127.0.0.1:47922)
0812 01:19:54.688177 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:54.688223 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:54.688288 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:54.688528 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:54.688531 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:54.688533 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:19:54.688606 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:54.688607 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:54.689731 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:54.689732 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:54.690918 (+ 1186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:54.690920 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:54.691845 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:54.691846 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:54.691941 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:54.691960 (+ 19us) client_negotiation.cc:769] Sending connection context
0812 01:19:54.692018 (+ 58us) client_negotiation.cc:240] Negotiation successful
0812 01:19:54.692109 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":72,"threads_started":1}
I20260812 01:19:54.692345 9744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:54.687971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.131:34925 (local address 127.0.0.1:42538)
0812 01:19:54.688177 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:54.688222 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:54.688294 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:54.688778 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:54.688779 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:54.688781 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:19:54.688831 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:54.688834 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:54.690158 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:54.690159 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:54.691196 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:54.691198 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:54.692116 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:54.692117 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:54.692176 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:54.692180 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 01:19:54.692231 (+ 51us) client_negotiation.cc:240] Negotiation successful
0812 01:19:54.692275 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":119,"threads_started":1}
I20260812 01:19:54.692580 9754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:54.688341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42538 (local address 127.24.0.131:34925)
0812 01:19:54.688529 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:54.688535 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:54.688546 (+ 11us) server_negotiation.cc:407] Connection header received
0812 01:19:54.688595 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:54.688597 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:54.688613 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:54.688695 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:19:54.688918 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:54.690034 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:54.691295 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:54.691930 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:54.692000 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:54.692058 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:54.692461 (+ 403us) server_negotiation.cc:299] Negotiation successful
0812 01:19:54.692510 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":115,"threads_started":1}
I20260812 01:19:54.692229 9748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:54.688052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47922 (local address 127.24.0.129:44909)
0812 01:19:54.688355 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:54.688361 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:54.688371 (+ 10us) server_negotiation.cc:407] Connection header received
0812 01:19:54.688416 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:54.688418 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:54.688430 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:54.688523 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:19:54.688724 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:54.689632 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:54.691021 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:54.691752 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:54.691827 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:54.692044 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:54.692096 (+ 52us) server_negotiation.cc:299] Negotiation successful
0812 01:19:54.692146 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":268,"thread_start_us":99,"threads_started":1}
I20260812 01:19:54.692716 9753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:54.688159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.130:44481 (local address 127.0.0.1:53954)
0812 01:19:54.688832 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:54.688844 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:54.688887 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:54.689171 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:54.689172 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:54.689174 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:19:54.689214 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:54.689215 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:54.690407 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:54.690408 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:54.691416 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:54.691419 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:54.692458 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:54.692459 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:54.692524 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:54.692528 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 01:19:54.692574 (+ 46us) client_negotiation.cc:240] Negotiation successful
0812 01:19:54.692615 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":655,"thread_start_us":85,"threads_started":1}
I20260812 01:19:54.693020 9755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:54.688656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53954 (local address 127.24.0.130:44481)
0812 01:19:54.689002 (+ 346us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:54.689007 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:54.689015 (+ 8us) server_negotiation.cc:407] Connection header received
0812 01:19:54.689052 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:54.689054 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:54.689068 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:54.689149 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:19:54.689297 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:54.690309 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:54.691508 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:54.692291 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:54.692365 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:54.692422 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:54.692875 (+ 453us) server_negotiation.cc:299] Negotiation successful
0812 01:19:54.692925 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":91,"threads_started":1}
I20260812 01:19:54.693336 9609 tablet_service.cc:1467] Processing CreateTablet for tablet e4e621b5fabe44928ac7b5eabd512f8c (DEFAULT_TABLE table=test-table [id=8949b8fc6d734b4d887fe464e9364093]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:19:54.693368 9122 tablet_service.cc:1467] Processing CreateTablet for tablet e4e621b5fabe44928ac7b5eabd512f8c (DEFAULT_TABLE table=test-table [id=8949b8fc6d734b4d887fe464e9364093]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:19:54.693581 9362 tablet_service.cc:1467] Processing CreateTablet for tablet e4e621b5fabe44928ac7b5eabd512f8c (DEFAULT_TABLE table=test-table [id=8949b8fc6d734b4d887fe464e9364093]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:19:54.693758 9122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4e621b5fabe44928ac7b5eabd512f8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:19:54.693761 9609 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4e621b5fabe44928ac7b5eabd512f8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:19:54.693909 9362 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4e621b5fabe44928ac7b5eabd512f8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:19:54.696671 9761 tablet_bootstrap.cc:492] T e4e621b5fabe44928ac7b5eabd512f8c P 3fb04fb44ec0473eb4cda10636008e23: Bootstrap starting.
I20260812 01:19:54.697183 9764 tablet_bootstrap.cc:492] T e4e621b5fabe44928ac7b5eabd512f8c P 76786ee43cc0428b91f4e96437817565: Bootstrap starting.
I20260812 01:19:54.697403 9767 tablet_bootstrap.cc:492] T e4e621b5fabe44928ac7b5eabd512f8c P 19b7bd8107cd45d2a440defafeb59ee2: Bootstrap starting.
I20260812 01:19:54.697484 9761 tablet_bootstrap.cc:654] T e4e621b5fabe44928ac7b5eabd512f8c P 3fb04fb44ec0473eb4cda10636008e23: Neither blocks nor log segments found. Creating new log.
I20260812 01:19:54.697883 9764 tablet_bootstrap.cc:654] T e4e621b5fabe44928ac7b5eabd512f8c P 76786ee43cc0428b91f4e96437817565: Neither blocks nor log segments found. Creating new log.
I20260812 01:19:54.697991 9767 tablet_bootstrap.cc:654] T e4e621b5fabe44928ac7b5eabd512f8c P 19b7bd8107cd45d2a440defafeb59ee2: Neither blocks nor log segments found. Creating new log.
I20260812 01:19:54.698117 9761 log.cc:824] T e4e621b5fabe44928ac7b5eabd512f8c P 3fb04fb44ec0473eb4cda10636008e23: Log is configured to *not* fsync() on all Append() calls
I20260812 01:19:54.698473 9764 log.cc:824] T e4e621b5fabe44928ac7b5eabd512f8c P 76786ee43cc0428b91f4e96437817565: Log is configured to *not* fsync() on all Append() calls
I20260812 01:19:54.698602 9767 log.cc:824] T e4e621b5fabe44928ac7b5eabd512f8c P 19b7bd8107cd45d2a440defafeb59ee2: Log is configured to *not* fsync() on all Append() calls
I20260812 01:19:54.699061 9761 tablet_bootstrap.cc:492] T e4e621b5fabe44928ac7b5eabd512f8c P 3fb04fb44ec0473eb4cda10636008e23: No bootstrap required, opened a new log
I20260812 01:19:54.699126 9761 ts_tablet_manager.cc:1397] T e4e621b5fabe44928ac7b5eabd512f8c P 3fb04fb44ec0473eb4cda10636008e23: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 01:19:54.699281 9764 tablet_bootstrap.cc:492] T e4e621b5fabe44928ac7b5eabd512f8c P 76786ee43cc0428b91f4e96437817565: No bootstrap required, opened a new log
I20260812 01:19:54.699361 9764 ts_tablet_manager.cc:1397] T e4e621b5fabe44928ac7b5eabd512f8c P 76786ee43cc0428b91f4e96437817565: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 01:19:54.700439 9767 tablet_bootstrap.cc:492] T e4e621b5fabe44928ac7b5eabd512f8c P 19b7bd8107cd45d2a440defafeb59ee2: No bootstrap required, opened a new log
I20260812 01:19:54.700513 9767 ts_tablet_manager.cc:1397] T e4e621b5fabe44928ac7b5eabd512f8c P 19b7bd8107cd45d2a440defafeb59ee2: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 01:19:54.700958 9761 raft_consensus.cc:348] T e4e621b5fabe44928ac7b5eabd512f8c P 3fb04fb44ec0473eb4cda10636008e23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b7bd8107cd45d2a440defafeb59ee2" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 44909 } } peers { permanent_uuid: "76786ee43cc0428b91f4e96437817565" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 44481 } } peers { permanent_uuid: "3fb04fb44ec0473eb4cda10636008e23" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34925 } }
I20260812 01:19:54.701121 9761 raft_consensus.cc:374] T e4e621b5fabe44928ac7b5eabd512f8c P 3fb04fb44ec0473eb4cda10636008e23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:19:54.701165 9761 raft_consensus.cc:729] T e4e621b5fabe44928ac7b5eabd512f8c P 3fb04fb44ec0473eb4cda10636008e23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fb04fb44ec0473eb4cda10636008e23, State: Initialized, Role: FOLLOWER
I20260812 01:19:54.701287 9761 consensus_queue.cc:260] T e4e621b5fabe44928ac7b5eabd512f8c P 3fb04fb44ec0473eb4cda10636008e23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b7bd8107cd45d2a440defafeb59ee2" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 44909 } } peers { permanent_uuid: "76786ee43cc0428b91f4e96437817565" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 44481 } } peers { permanent_uuid: "3fb04fb44ec0473eb4cda10636008e23" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34925 } }
I20260812 01:19:54.701397 9764 raft_consensus.cc:348] T e4e621b5fabe44928ac7b5eabd512f8c P 76786ee43cc0428b91f4e96437817565 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b7bd8107cd45d2a440defafeb59ee2" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 44909 } } peers { permanent_uuid: "76786ee43cc0428b91f4e96437817565" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 44481 } } peers { permanent_uuid: "3fb04fb44ec0473eb4cda10636008e23" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34925 } }
I20260812 01:19:54.701565 9764 raft_consensus.cc:374] T e4e621b5fabe44928ac7b5eabd512f8c P 76786ee43cc0428b91f4e96437817565 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:19:54.701606 9764 raft_consensus.cc:729] T e4e621b5fabe44928ac7b5eabd512f8c P 76786ee43cc0428b91f4e96437817565 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76786ee43cc0428b91f4e96437817565, State: Initialized, Role: FOLLOWER
I20260812 01:19:54.701597 9761 ts_tablet_manager.cc:1428] T e4e621b5fabe44928ac7b5eabd512f8c P 3fb04fb44ec0473eb4cda10636008e23: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 01:19:54.701673 9732 heartbeater.cc:503] Master 127.24.0.190:35681 was elected leader, sending a full tablet report...
I20260812 01:19:54.701719 9764 consensus_queue.cc:260] T e4e621b5fabe44928ac7b5eabd512f8c P 76786ee43cc0428b91f4e96437817565 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b7bd8107cd45d2a440defafeb59ee2" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 44909 } } peers { permanent_uuid: "76786ee43cc0428b91f4e96437817565" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 44481 } } peers { permanent_uuid: "3fb04fb44ec0473eb4cda10636008e23" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34925 } }
I20260812 01:19:54.701998 9472 heartbeater.cc:503] Master 127.24.0.190:35681 was elected leader, sending a full tablet report...
I20260812 01:19:54.701983 9764 ts_tablet_manager.cc:1428] T e4e621b5fabe44928ac7b5eabd512f8c P 76786ee43cc0428b91f4e96437817565: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260812 01:19:54.702723 9767 raft_consensus.cc:348] T e4e621b5fabe44928ac7b5eabd512f8c P 19b7bd8107cd45d2a440defafeb59ee2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b7bd8107cd45d2a440defafeb59ee2" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 44909 } } peers { permanent_uuid: "76786ee43cc0428b91f4e96437817565" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 44481 } } peers { permanent_uuid: "3fb04fb44ec0473eb4cda10636008e23" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34925 } }
I20260812 01:19:54.702883 9767 raft_consensus.cc:374] T e4e621b5fabe44928ac7b5eabd512f8c P 19b7bd8107cd45d2a440defafeb59ee2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:19:54.702929 9767 raft_consensus.cc:729] T e4e621b5fabe44928ac7b5eabd512f8c P 19b7bd8107cd45d2a440defafeb59ee2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19b7bd8107cd45d2a440defafeb59ee2, State: Initialized, Role: FOLLOWER
I20260812 01:19:54.703035 9767 consensus_queue.cc:260] T e4e621b5fabe44928ac7b5eabd512f8c P 19b7bd8107cd45d2a440defafeb59ee2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b7bd8107cd45d2a440defafeb59ee2" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 44909 } } peers { permanent_uuid: "76786ee43cc0428b91f4e96437817565" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 44481 } } peers { permanent_uuid: "3fb04fb44ec0473eb4cda10636008e23" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34925 } }
I20260812 01:19:54.703343 9767 ts_tablet_manager.cc:1428] T e4e621b5fabe44928ac7b5eabd512f8c P 19b7bd8107cd45d2a440defafeb59ee2: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 01:19:54.703436 9240 heartbeater.cc:503] Master 127.24.0.190:35681 was elected leader, sending a full tablet report...
W20260812 01:19:54.742976 9473 tablet.cc:2369] T e4e621b5fabe44928ac7b5eabd512f8c P 76786ee43cc0428b91f4e96437817565: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 01:19:54.817799 9775 raft_consensus.cc:482] T e4e621b5fabe44928ac7b5eabd512f8c P 76786ee43cc0428b91f4e96437817565 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 01:19:54.817924 9775 raft_consensus.cc:504] T e4e621b5fabe44928ac7b5eabd512f8c P 76786ee43cc0428b91f4e96437817565 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b7bd8107cd45d2a440defafeb59ee2" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 44909 } } peers { permanent_uuid: "76786ee43cc0428b91f4e96437817565" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 44481 } } peers { permanent_uuid: "3fb04fb44ec0473eb4cda10636008e23" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34925 } }
I20260812 01:19:54.818379 9775 leader_election.cc:288] T e4e621b5fabe44928ac7b5eabd512f8c P 76786ee43cc0428b91f4e96437817565 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19b7bd8107cd45d2a440defafeb59ee2 (127.24.0.129:44909), 3fb04fb44ec0473eb4cda10636008e23 (127.24.0.131:34925)
I20260812 01:19:54.822304 9748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:54.818707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.130:52393 (local address 127.24.0.129:44909)
0812 01:19:54.818755 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:54.818757 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:54.819054 (+ 297us) server_negotiation.cc:407] Connection header received
0812 01:19:54.819107 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:54.819109 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:54.819123 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:54.819186 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:19:54.819317 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:54.820029 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:54.821335 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:54.821906 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:54.821989 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:54.821999 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:54.822188 (+ 189us) server_negotiation.cc:299] Negotiation successful
0812 01:19:54.822228 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 01:19:54.822304 9754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:54.818703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.130:55381 (local address 127.24.0.131:34925)
0812 01:19:54.818777 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:54.818780 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:54.818791 (+ 11us) server_negotiation.cc:407] Connection header received
0812 01:19:54.818826 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:54.818827 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:54.818841 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:54.818916 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:19:54.819113 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:54.819896 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:54.821335 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:54.821906 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:54.821989 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:54.821999 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:54.822188 (+ 189us) server_negotiation.cc:299] Negotiation successful
0812 01:19:54.822228 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 01:19:54.822309 9476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:54.818614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.131:34925 (local address 127.24.0.130:55381)
0812 01:19:54.818679 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:54.818693 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:54.818762 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:54.818936 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:54.818937 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:54.818939 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:19:54.819024 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:54.819026 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:54.819975 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:54.819976 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:54.821223 (+ 1247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:54.821227 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:54.821999 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:54.822000 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:54.822110 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:54.822112 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 01:19:54.822166 (+ 54us) client_negotiation.cc:240] Negotiation successful
0812 01:19:54.822210 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":11}
I20260812 01:19:54.822314 9778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:54.818614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.129:44909 (local address 127.24.0.130:52393)
0812 01:19:54.818965 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:54.818976 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 01:19:54.819038 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:19:54.819196 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:19:54.819196 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:19:54.819198 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:19:54.819246 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:54.819247 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:54.820157 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:54.820158 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:54.821245 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:19:54.821246 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:19:54.821999 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:19:54.822000 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:19:54.822091 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:54.822096 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 01:19:54.822160 (+ 64us) client_negotiation.cc:240] Negotiation successful
0812 01:19:54.822209 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"mutex_wait_us":145,"thread_start_us":131,"threads_started":1}
I20260812 01:19:54.822749 9644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4e621b5fabe44928ac7b5eabd512f8c" candidate_uuid: "76786ee43cc0428b91f4e96437817565" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fb04fb44ec0473eb4cda10636008e23" is_pre_election: true
I20260812 01:19:54.822757 9161 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4e621b5fabe44928ac7b5eabd512f8c" candidate_uuid: "76786ee43cc0428b91f4e96437817565" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b7bd8107cd45d2a440defafeb59ee2" is_pre_election: true
I20260812 01:19:54.822924 9644 raft_consensus.cc:2454] T e4e621b5fabe44928ac7b5eabd512f8c P 3fb04fb44ec0473eb4cda10636008e23 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76786ee43cc0428b91f4e96437817565 in term 0.
I20260812 01:19:54.822924 9161 raft_consensus.cc:2454] T e4e621b5fabe44928ac7b5eabd512f8c P 19b7bd8107cd45d2a440defafeb59ee2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76786ee43cc0428b91f4e96437817565 in term 0.
I20260812 01:19:54.823176 9277 leader_election.cc:302] T e4e621b5fabe44928ac7b5eabd512f8c P 76786ee43cc0428b91f4e96437817565 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19b7bd8107cd45d2a440defafeb59ee2, 76786ee43cc0428b91f4e96437817565; no voters:
I20260812 01:19:54.823324 9775 raft_consensus.cc:2781] T e4e621b5fabe44928ac7b5eabd512f8c P 76786ee43cc0428b91f4e96437817565 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 01:19:54.823385 9775 raft_consensus.cc:482] T e4e621b5fabe44928ac7b5eabd512f8c P 76786ee43cc0428b91f4e96437817565 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 01:19:54.823410 9775 raft_consensus.cc:3037] T e4e621b5fabe44928ac7b5eabd512f8c P 76786ee43cc0428b91f4e96437817565 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:19:54.824244 9775 raft_consensus.cc:504] T e4e621b5fabe44928ac7b5eabd512f8c P 76786ee43cc0428b91f4e96437817565 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b7bd8107cd45d2a440defafeb59ee2" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 44909 } } peers { permanent_uuid: "76786ee43cc0428b91f4e96437817565" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 44481 } } peers { permanent_uuid: "3fb04fb44ec0473eb4cda10636008e23" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34925 } }
I20260812 01:19:54.824397 9775 leader_election.cc:288] T e4e621b5fabe44928ac7b5eabd512f8c P 76786ee43cc0428b91f4e96437817565 [CANDIDATE]: Term 1 election: Requested vote from peers 19b7bd8107cd45d2a440defafeb59ee2 (127.24.0.129:44909), 3fb04fb44ec0473eb4cda10636008e23 (127.24.0.131:34925)
I20260812 01:19:54.824597 9161 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4e621b5fabe44928ac7b5eabd512f8c" candidate_uuid: "76786ee43cc0428b91f4e96437817565" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b7bd8107cd45d2a440defafeb59ee2"
I20260812 01:19:54.824623 9644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4e621b5fabe44928ac7b5eabd512f8c" candidate_uuid: "76786ee43cc0428b91f4e96437817565" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fb04fb44ec0473eb4cda10636008e23"
I20260812 01:19:54.824693 9644 raft_consensus.cc:3037] T e4e621b5fabe44928ac7b5eabd512f8c P 3fb04fb44ec0473eb4cda10636008e23 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:19:54.824699 9161 raft_consensus.cc:3037] T e4e621b5fabe44928ac7b5eabd512f8c P 19b7bd8107cd45d2a440defafeb59ee2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:19:54.825438 9161 raft_consensus.cc:2454] T e4e621b5fabe44928ac7b5eabd512f8c P 19b7bd8107cd45d2a440defafeb59ee2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76786ee43cc0428b91f4e96437817565 in term 1.
I20260812 01:19:54.825515 9644 raft_consensus.cc:2454] T e4e621b5fabe44928ac7b5eabd512f8c P 3fb04fb44ec0473eb4cda10636008e23 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76786ee43cc0428b91f4e96437817565 in term 1.
I20260812 01:19:54.825654 9277 leader_election.cc:302] T e4e621b5fabe44928ac7b5eabd512f8c P 76786ee43cc0428b91f4e96437817565 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19b7bd8107cd45d2a440defafeb59ee2, 76786ee43cc0428b91f4e96437817565; no voters:
I20260812 01:19:54.825743 9775 raft_consensus.cc:2781] T e4e621b5fabe44928ac7b5eabd512f8c P 76786ee43cc0428b91f4e96437817565 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:19:54.825904 9775 raft_consensus.cc:686] T e4e621b5fabe44928ac7b5eabd512f8c P 76786ee43cc0428b91f4e96437817565 [term 1 LEADER]: Becoming Leader. State: Replica: 76786ee43cc0428b91f4e96437817565, State: Running, Role: LEADER
I20260812 01:19:54.826009 9775 consensus_queue.cc:237] T e4e621b5fabe44928ac7b5eabd512f8c P 76786ee43cc0428b91f4e96437817565 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b7bd8107cd45d2a440defafeb59ee2" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 44909 } } peers { permanent_uuid: "76786ee43cc0428b91f4e96437817565" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 44481 } } peers { permanent_uuid: "3fb04fb44ec0473eb4cda10636008e23" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34925 } }
W20260812 01:19:54.826959 9241 tablet.cc:2369] T e4e621b5fabe44928ac7b5eabd512f8c P 19b7bd8107cd45d2a440defafeb59ee2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 01:19:54.827046 8896 catalog_manager.cc:5495] T e4e621b5fabe44928ac7b5eabd512f8c P 76786ee43cc0428b91f4e96437817565 reported cstate change: term changed from 0 to 1, leader changed from <none> to 76786ee43cc0428b91f4e96437817565 (127.24.0.130). New cstate: current_term: 1 leader_uuid: "76786ee43cc0428b91f4e96437817565" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b7bd8107cd45d2a440defafeb59ee2" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 44909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76786ee43cc0428b91f4e96437817565" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 44481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3fb04fb44ec0473eb4cda10636008e23" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34925 } health_report { overall_health: UNKNOWN } } }
I20260812 01:19:54.837363 9755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:54.835075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53968 (local address 127.24.0.130:44481)
0812 01:19:54.835123 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 01:19:54.835125 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:19:54.835133 (+ 8us) server_negotiation.cc:407] Connection header received
0812 01:19:54.835194 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:19:54.835195 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:19:54.835213 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:19:54.835284 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:19:54.835380 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:54.835846 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:54.836639 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:19:54.836830 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:19:54.836894 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:19:54.837012 (+ 118us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:19:54.837109 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:19:54.837166 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 01:19:54.837225 (+ 59us) server_negotiation.cc:299] Negotiation successful
0812 01:19:54.837247 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 01:19:54.839138 9161 raft_consensus.cc:1261] T e4e621b5fabe44928ac7b5eabd512f8c P 19b7bd8107cd45d2a440defafeb59ee2 [term 1 FOLLOWER]: Refusing update from remote peer 76786ee43cc0428b91f4e96437817565: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:19:54.839151 9644 raft_consensus.cc:1261] T e4e621b5fabe44928ac7b5eabd512f8c P 3fb04fb44ec0473eb4cda10636008e23 [term 1 FOLLOWER]: Refusing update from remote peer 76786ee43cc0428b91f4e96437817565: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:19:54.839479 9779 consensus_queue.cc:1035] T e4e621b5fabe44928ac7b5eabd512f8c P 76786ee43cc0428b91f4e96437817565 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19b7bd8107cd45d2a440defafeb59ee2" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 44909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:19:54.839625 9775 consensus_queue.cc:1035] T e4e621b5fabe44928ac7b5eabd512f8c P 76786ee43cc0428b91f4e96437817565 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fb04fb44ec0473eb4cda10636008e23" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:19:54.842216 9786 mvcc.cc:205] Tried to move back new op lower bound from 7317494148458651648 to 7317494148408037376. Current Snapshot: MvccSnapshot[applied={T|T < 7317494148458651648}]
I20260812 01:19:54.842335 9788 mvcc.cc:205] Tried to move back new op lower bound from 7317494148458651648 to 7317494148408037376. Current Snapshot: MvccSnapshot[applied={T|T < 7317494148458651648}]
I20260812 01:19:54.844789 8896 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33552:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:19:54.844893 8896 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33552:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:19:54.845757 8896 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8d7424c39e3b4743a2e09f70dbfb5b2a: Sending DeleteTablet for 3 replicas of tablet e4e621b5fabe44928ac7b5eabd512f8c
I20260812 01:19:54.846129 9609 tablet_service.cc:1514] Processing DeleteTablet for tablet e4e621b5fabe44928ac7b5eabd512f8c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:19:54 UTC) from {username='kudu', principal='kudu/127.24.0.190@KRBTEST.COM'} at 127.0.0.1:42538
I20260812 01:19:54.846151 9122 tablet_service.cc:1514] Processing DeleteTablet for tablet e4e621b5fabe44928ac7b5eabd512f8c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:19:54 UTC) from {username='kudu', principal='kudu/127.24.0.190@KRBTEST.COM'} at 127.0.0.1:47922
I20260812 01:19:54.846383 9801 tablet_replica.cc:330] T e4e621b5fabe44928ac7b5eabd512f8c P 19b7bd8107cd45d2a440defafeb59ee2: stopping tablet replica
I20260812 01:19:54.846483 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 9018
I20260812 01:19:54.857533 9362 tablet_service.cc:1514] Processing DeleteTablet for tablet e4e621b5fabe44928ac7b5eabd512f8c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:19:54 UTC) from {username='kudu', principal='kudu/127.24.0.190@KRBTEST.COM'} at 127.0.0.1:53954
I20260812 01:19:54.858238 9802 tablet_replica.cc:330] T e4e621b5fabe44928ac7b5eabd512f8c P 76786ee43cc0428b91f4e96437817565: stopping tablet replica
I20260812 01:19:54.858409 9802 raft_consensus.cc:2229] T e4e621b5fabe44928ac7b5eabd512f8c P 76786ee43cc0428b91f4e96437817565 [term 1 LEADER]: Raft consensus shutting down.
I20260812 01:19:54.858621 9802 raft_consensus.cc:2258] T e4e621b5fabe44928ac7b5eabd512f8c P 76786ee43cc0428b91f4e96437817565 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:19:54.859342 9802 ts_tablet_manager.cc:1905] T e4e621b5fabe44928ac7b5eabd512f8c P 76786ee43cc0428b91f4e96437817565: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:19:54.861321 9802 ts_tablet_manager.cc:1918] T e4e621b5fabe44928ac7b5eabd512f8c P 76786ee43cc0428b91f4e96437817565: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 01:19:54.861476 9802 log.cc:1196] T e4e621b5fabe44928ac7b5eabd512f8c P 76786ee43cc0428b91f4e96437817565: Deleting WAL directory at /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786497534195564-24578-0/minicluster-data/ts-1/wal/wals/e4e621b5fabe44928ac7b5eabd512f8c
I20260812 01:19:54.861860 9802 ts_tablet_manager.cc:1939] T e4e621b5fabe44928ac7b5eabd512f8c P 76786ee43cc0428b91f4e96437817565: Deleting consensus metadata
I20260812 01:19:54.862793 8869 catalog_manager.cc:4841] TS 76786ee43cc0428b91f4e96437817565 (127.24.0.130:44481): tablet e4e621b5fabe44928ac7b5eabd512f8c (table test-table [id=8949b8fc6d734b4d887fe464e9364093]) successfully deleted
W20260812 01:19:54.871412 8869 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.0.129:44909 (error 108)
I20260812 01:19:54.871558 9797 tablet_replica.cc:330] T e4e621b5fabe44928ac7b5eabd512f8c P 3fb04fb44ec0473eb4cda10636008e23: stopping tablet replica
I20260812 01:19:54.871665 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 9247
I20260812 01:19:54.871714 9797 raft_consensus.cc:2229] T e4e621b5fabe44928ac7b5eabd512f8c P 3fb04fb44ec0473eb4cda10636008e23 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 01:19:54.871869 9797 raft_consensus.cc:2258] T e4e621b5fabe44928ac7b5eabd512f8c P 3fb04fb44ec0473eb4cda10636008e23 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:19:54.877664 9797 ts_tablet_manager.cc:1905] T e4e621b5fabe44928ac7b5eabd512f8c P 3fb04fb44ec0473eb4cda10636008e23: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:19:54.879632 9797 ts_tablet_manager.cc:1918] T e4e621b5fabe44928ac7b5eabd512f8c P 3fb04fb44ec0473eb4cda10636008e23: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 01:19:54.879715 9753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:19:54.879483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.129:44909 (local address 127.0.0.1:47926)
0812 01:19:54.879604 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0812 01:19:54.879665 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.129:44909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260812 01:19:54.879747 9797 log.cc:1196] T e4e621b5fabe44928ac7b5eabd512f8c P 3fb04fb44ec0473eb4cda10636008e23: Deleting WAL directory at /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786497534195564-24578-0/minicluster-data/ts-2/wal/wals/e4e621b5fabe44928ac7b5eabd512f8c
W20260812 01:19:54.879887 8869 catalog_manager.cc:4584] TS 19b7bd8107cd45d2a440defafeb59ee2 (127.24.0.129:44909): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e4e621b5fabe44928ac7b5eabd512f8c: Network error: Client connection negotiation failed: client connection to 127.24.0.129:44909: connect: Connection refused (error 111)
I20260812 01:19:54.880107 9797 ts_tablet_manager.cc:1939] T e4e621b5fabe44928ac7b5eabd512f8c P 3fb04fb44ec0473eb4cda10636008e23: Deleting consensus metadata
I20260812 01:19:54.880618 8874 catalog_manager.cc:4841] TS 3fb04fb44ec0473eb4cda10636008e23 (127.24.0.131:34925): tablet e4e621b5fabe44928ac7b5eabd512f8c (table test-table [id=8949b8fc6d734b4d887fe464e9364093]) successfully deleted
I20260812 01:19:54.883857 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 9485
I20260812 01:19:54.898031 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 8843
2026-08-12T01:19:54Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2330 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 01:19:54 dist-test-slave-t3q3 krb5kdc[9809](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 01:19:54 dist-test-slave-t3q3 krb5kdc[9809](info): set up 2 sockets
Aug 12 01:19:54 dist-test-slave-t3q3 krb5kdc[9809](info): commencing operation
krb5kdc: starting...
W20260812 01:19:56.177265 24578 mini_kdc.cc:121] Time spent starting KDC: real 1.251s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/krb5kdc/test-user.keytab.
Aug 12 01:19:56 dist-test-slave-t3q3 krb5kdc[9809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497596, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T01:19:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:19:56Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.24.0.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.24.0.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.24.0.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/krb5kdc/rangeradmin_127.24.0.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.24.0.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/krb5kdc/rangeradmin_127.24.0.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.24.0.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.24.0.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.24.0.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/krb5kdc/rangerlookup_127.24.0.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.24.0.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/krb5kdc/rangerlookup_127.24.0.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.24.0.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.24.0.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.24.0.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/krb5kdc/HTTP_127.24.0.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.24.0.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/krb5kdc/HTTP_127.24.0.148@KRBTEST.COM.keytab.
I20260812 01:19:56.361125 24578 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskvoD5sF/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/minicluster-data/postgres -l logfile start
2026-08-12 01:19:58.894 UTC [9867] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-12 01:19:58.894 UTC [9867] LOG: listening on IPv4 address "127.24.0.148", port 48847
2026-08-12 01:19:58.898 UTC [9867] LOG: listening on Unix socket "/tmp/.s.PGSQL.48847"
2026-08-12 01:19:58.920 UTC [9872] LOG: database system was shut down at 2026-08-12 01:19:57 UTC
2026-08-12 01:19:58.924 UTC [9867] LOG: database system is ready to accept connections
I20260812 01:20:00.132627 24578 mini_postgres.cc:96] Postgres bound to 48847
2026-08-12 01:20:00.141 UTC [9889] FATAL: database "slave" does not exist
127.24.0.148:48847 - accepting connections
I20260812 01:20:00.142370 24578 mini_ranger.cc:161] Starting Ranger...
I20260812 01:20:00.158880 24578 mini_ranger.cc:84] Created miniranger Postgres user
I20260812 01:20:00.593216 24578 mini_ranger.cc:87] Created ranger Postgres database
I20260812 01:20:00.593309 24578 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/minicluster-data/ranger-admin
2026-08-12 01:20:00,989 [I] DB FLAVOR :POSTGRES
2026-08-12 01:20:00,990 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 01:20:00,990 [I] Checking connection..
2026-08-12 01:20:00,990 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvoD5sF/build/release/bin/postgresql.jar:/tmp/dist-test-taskvoD5sF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.0.148:48847/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-12 01:20:01,287 [I] Checking connection passed.
2026-08-12 01:20:01,287 [I] --------- Verifying version history table ---------
2026-08-12 01:20:01,288 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvoD5sF/build/release/bin/postgresql.jar:/tmp/dist-test-taskvoD5sF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.0.148:48847/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 01:20:01,555 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 01:20:01,556 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvoD5sF/build/release/bin/postgresql.jar:/tmp/dist-test-taskvoD5sF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.0.148:48847/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 01:20:01,811 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 01:20:01,811 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-12 01:20:01,811 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvoD5sF/build/release/bin/postgresql.jar:/tmp/dist-test-taskvoD5sF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.0.148:48847/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskvoD5sF/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-12 01:20:02.075 UTC [10013] WARNING: there is no transaction in progress
2026-08-12 01:20:02,084 [I] create_dbversion_catalog.sql file imported successfully
2026-08-12 01:20:02,085 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvoD5sF/build/release/bin/postgresql.jar:/tmp/dist-test-taskvoD5sF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.0.148:48847/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 01:20:02,355 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-12 01:20:02,355 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 01:20:02,355 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvoD5sF/build/release/bin/postgresql.jar:/tmp/dist-test-taskvoD5sF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.0.148:48847/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-12 01:20:02,618 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvoD5sF/build/release/bin/postgresql.jar:/tmp/dist-test-taskvoD5sF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.0.148:48847/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-12 01:20:02,891 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvoD5sF/build/release/bin/postgresql.jar:/tmp/dist-test-taskvoD5sF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.0.148:48847/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-t3q3.c.gcp-upstream.internal','N') ;"
2026-08-12 01:20:03,155 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvoD5sF/build/release/bin/postgresql.jar:/tmp/dist-test-taskvoD5sF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.0.148:48847/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 01:20:03,437 [I] Table x_portal_user does not exist in database ranger
2026-08-12 01:20:03,438 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvoD5sF/build/release/bin/postgresql.jar:/tmp/dist-test-taskvoD5sF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.0.148:48847/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 01:20:03,692 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-12 01:20:03,693 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvoD5sF/build/release/bin/postgresql.jar:/tmp/dist-test-taskvoD5sF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.0.148:48847/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 01:20:03,956 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvoD5sF/build/release/bin/postgresql.jar:/tmp/dist-test-taskvoD5sF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.0.148:48847/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 01:20:04,216 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-12 01:20:04,216 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvoD5sF/build/release/bin/postgresql.jar:/tmp/dist-test-taskvoD5sF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.0.148:48847/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskvoD5sF/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-12 01:20:05.187 UTC [10213] WARNING: there is no transaction in progress
2026-08-12 01:20:05.204 UTC [10213] WARNING: there is no transaction in progress
2026-08-12 01:20:05.221 UTC [10213] WARNING: there is no transaction in progress
2026-08-12 01:20:05.236 UTC [10213] WARNING: there is no transaction in progress
2026-08-12 01:20:05.251 UTC [10213] WARNING: there is no transaction in progress
2026-08-12 01:20:05.268 UTC [10213] WARNING: there is no transaction in progress
2026-08-12 01:20:05.323 UTC [10213] WARNING: there is no transaction in progress
2026-08-12 01:20:05.330 UTC [10213] WARNING: there is no transaction in progress
2026-08-12 01:20:05.345 UTC [10213] WARNING: there is no transaction in progress
2026-08-12 01:20:05.355 UTC [10213] WARNING: there is no transaction in progress
2026-08-12 01:20:05.365 UTC [10213] WARNING: there is no transaction in progress
2026-08-12 01:20:05.378 UTC [10213] WARNING: there is no transaction in progress
2026-08-12 01:20:05.387 UTC [10213] WARNING: there is no transaction in progress
2026-08-12 01:20:05.394 UTC [10213] WARNING: there is no transaction in progress
2026-08-12 01:20:05.400 UTC [10213] WARNING: there is no transaction in progress
2026-08-12 01:20:05.408 UTC [10213] WARNING: there is no transaction in progress
2026-08-12 01:20:06,051 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-12 01:20:06,052 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvoD5sF/build/release/bin/postgresql.jar:/tmp/dist-test-taskvoD5sF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.0.148:48847/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-12 01:20:06,299 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-12 01:20:06,300 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvoD5sF/build/release/bin/postgresql.jar:/tmp/dist-test-taskvoD5sF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.0.148:48847/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 01:20:06,578 [I] Table x_portal_user already exists in database 'ranger'
2026-08-12 01:20:06,578 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvoD5sF/build/release/bin/postgresql.jar:/tmp/dist-test-taskvoD5sF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.0.148:48847/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 01:20:06,848 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-12 01:20:06,848 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvoD5sF/build/release/bin/postgresql.jar:/tmp/dist-test-taskvoD5sF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.0.148:48847/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 01:20:07,099 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvoD5sF/build/release/bin/postgresql.jar:/tmp/dist-test-taskvoD5sF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.0.148:48847/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 01:20:07,370 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvoD5sF/build/release/bin/postgresql.jar:/tmp/dist-test-taskvoD5sF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.0.148:48847/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-t3q3.c.gcp-upstream.internal';"
2026-08-12 01:20:07,631 [I] CORE_DB_SCHEMA import status has been updated
2026-08-12 01:20:07,631 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvoD5sF/build/release/bin/postgresql.jar:/tmp/dist-test-taskvoD5sF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.0.148:48847/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-12 01:20:07,871 [I] DB_PATCHES have already been applied
I20260812 01:20:07.877808 24578 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/minicluster-data/ranger-admin:/tmp/dist-test-taskvoD5sF/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskvoD5sF/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskvoD5sF/build/release/bin/postgresql.jar:/tmp/dist-test-taskvoD5sF/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260812 01:20:07.877921 24578 mini_ranger.cc:193] Using host: 127.24.0.148
I20260812 01:20:07.882158 24578 mini_ranger.cc:227] Ranger admin URL: http://127.24.0.148:55563
Aug 12, 2026 1:20:08 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 1:20:08 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 1:20:08 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/minicluster-data/ranger-admin/ews/webapp
Aug 12, 2026 1:20:08 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 12, 2026 1:20:08 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 12, 2026 1:20:08 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/minicluster-data/ranger-admin/ews/webapp].
Aug 12, 2026 1:20:08 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260812 01:20:09.311900 24578 mini_ranger.cc:160] Time spent starting Ranger: real 9.170s user 0.000s sys 0.012s
Aug 12, 2026 1:20:29 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 12, 2026 1:20:29 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Aug 12, 2026 1:20:29 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 12, 2026 1:20:29 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 12, 2026 1:20:29 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 12, 2026 1:20:29 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 12, 2026 1:20:29 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 12, 2026 1:20:29 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 12, 2026 1:20:29 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 12, 2026 1:20:29 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 12, 2026 1:20:29 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 12, 2026 1:20:29 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 12, 2026 1:20:29 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 12, 2026 1:20:29 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 12, 2026 1:20:29 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 12, 2026 1:20:29 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 12, 2026 1:20:29 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 12, 2026 1:20:30 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260812 01:20:30.919130 24578 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.24.0.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.24.0.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.24.0.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/krb5kdc/rangerkms_127.24.0.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.24.0.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/krb5kdc/rangerkms_127.24.0.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.24.0.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.24.0.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.24.0.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/krb5kdc/HTTP_127.24.0.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.24.0.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/krb5kdc/HTTP_127.24.0.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 12 01:20:31 dist-test-slave-t3q3 krb5kdc[9809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497631, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260812 01:20:31.010358 24578 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260812 01:20:31.022324 24578 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260812 01:20:31.844749 24578 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260812 01:20:31.844854 24578 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/minicluster-data/ranger-kms
I20260812 01:20:31.844877 24578 mini_ranger_kms.cc:227] Using postgres at 127.24.0.148:48847
2026-08-12 01:20:32,238 [I] DB FLAVOR :POSTGRES
2026-08-12 01:20:32,238 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 01:20:32,238 [I] Checking connection
2026-08-12 01:20:32,238 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvoD5sF/build/release/bin/postgresql.jar:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.0.148:48847/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-12 01:20:32,496 [I] connection success
2026-08-12 01:20:32,496 [I] --------- Verifying Ranger DB tables ---------
2026-08-12 01:20:32,496 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-12 01:20:32,496 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvoD5sF/build/release/bin/postgresql.jar:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.0.148:48847/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-12 01:20:32,744 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-12 01:20:32,744 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 01:20:32,744 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-12 01:20:32,745 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvoD5sF/build/release/bin/postgresql.jar:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.0.148:48847/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-12 01:20:33,029 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260812 01:20:33.524245 24578 mini_ranger_kms.cc:310] Created kms service
I20260812 01:20:33.697744 24578 mini_ranger_kms.cc:326] Created kudu user
I20260812 01:20:33.740464 24578 mini_ranger_kms.cc:343] Created rangerkms user
Aug 12 01:20:33 dist-test-slave-t3q3 krb5kdc[9809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497631, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.24.0.148@KRBTEST.COM
I20260812 01:20:34.120445 24578 mini_ranger_kms.cc:384] Added ranger policy
I20260812 01:20:34.120579 24578 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/minicluster-data/ranger-kms:/tmp/dist-test-taskvoD5sF/build/release/bin/postgresql.jar:/tmp/dist-test-taskvoD5sF/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskvoD5sF/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260812 01:20:34.120607 24578 mini_ranger_kms.cc:240] Using host: 127.24.0.148
I20260812 01:20:34.125761 24578 mini_ranger_kms.cc:276] Ranger KMS PID: 10573
I20260812 01:20:34.125890 24578 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.24.0.148:33709
Aug 12, 2026 1:20:35 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 1:20:35 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 1:20:35 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 12, 2026 1:20:35 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 12, 2026 1:20:35 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/minicluster-data/ranger-kms/ews/webapp].
Aug 12, 2026 1:20:35 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 12 01:20:36 dist-test-slave-t3q3 krb5kdc[9809](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786497636, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.24.0.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:20:36.931463 24578 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.921s user 0.005s sys 0.227s
I20260812 01:20:36.931568 24578 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260812 01:20:36.931644 24578 mini_ranger_kms.cc:401] 127.24.0.148:33709/kms/v1/keys
Aug 12 01:20:37 dist-test-slave-t3q3 krb5kdc[9809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497636, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.24.0.148@KRBTEST.COM for HTTP/127.24.0.148@KRBTEST.COM
Aug 12 01:20:37 dist-test-slave-t3q3 krb5kdc[9809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497636, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.24.0.148@KRBTEST.COM for HTTP/127.24.0.148@KRBTEST.COM
Aug 12 01:20:39 dist-test-slave-t3q3 krb5kdc[9809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497639, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.24.0.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.0.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.0.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab.
Entry for principal kudu/127.24.0.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab.
I20260812 01:20:39.911167 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.190:34847
--webserver_interface=127.24.0.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab
--principal=kudu/127.24.0.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:44139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.0.190:34847
--ranger_config_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.0.148:33709/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:20:40.007761 10659 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:20:40.007972 10659 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:20:40.007994 10659 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:20:40.009157 10659 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:20:40.009202 10659 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:20:40.009217 10659 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:20:40.009229 10659 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:20:40.009241 10659 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:20:40.010838 10659 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:44139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/minicluster-data/master-0/wal
--ranger_kms_url=127.24.0.148:33709/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.24.0.190:34847
--ranger_config_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab
--principal=kudu/127.24.0.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.190:34847
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.0.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10659
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:20:40.011041 10659 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:20:40.011217 10659 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:20:40.016551 10668 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:20:40.016511 10674 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:20:40.016562 10659 server_base.cc:1034] running on GCE node
W20260812 01:20:40.016531 10666 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:20:40.016924 10659 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:20:40.017246 10659 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:20:40.018427 10659 hybrid_clock.cc:648] HybridClock initialized: now 1786497640018413 us; error 46 us; skew 500 ppm
Aug 12 01:20:40 dist-test-slave-t3q3 krb5kdc[9809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497640, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:20:40.025269 10659 init.cc:377] Logged in from keytab as kudu/127.24.0.190@KRBTEST.COM (short username kudu)
I20260812 01:20:40.025756 10659 webserver.cc:460] Webserver started at http://127.24.0.190:42813/ using document root <none> and password file <none>
I20260812 01:20:40.025993 10659 fs_manager.cc:359] Metadata directory not provided
I20260812 01:20:40.026068 10659 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:20:40.026234 10659 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 01:20:40 dist-test-slave-t3q3 krb5kdc[9809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497640, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.190@KRBTEST.COM for HTTP/127.24.0.148@KRBTEST.COM
I20260812 01:20:40.080149 10659 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/minicluster-data/master-0/data/instance:
uuid: "db2d2452aea54ae0b06a10744aa3e057"
format_stamp: "Formatted at 2026-08-12 01:20:40 on dist-test-slave-t3q3"
server_key: "b7f23c38d3707431258c1287f89d26c6"
server_key_iv: "23cdf77016e05de1a67e5c83379b977a"
server_key_version: "kuduclusterkey@0"
I20260812 01:20:40.080610 10659 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/minicluster-data/master-0/wal/instance:
uuid: "db2d2452aea54ae0b06a10744aa3e057"
format_stamp: "Formatted at 2026-08-12 01:20:40 on dist-test-slave-t3q3"
server_key: "b7f23c38d3707431258c1287f89d26c6"
server_key_iv: "23cdf77016e05de1a67e5c83379b977a"
server_key_version: "kuduclusterkey@0"
I20260812 01:20:40.082167 10659 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 01:20:40.094396 10688 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:20:40.094597 10659 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 01:20:40.094715 10659 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/minicluster-data/master-0/wal
uuid: "db2d2452aea54ae0b06a10744aa3e057"
format_stamp: "Formatted at 2026-08-12 01:20:40 on dist-test-slave-t3q3"
server_key: "b7f23c38d3707431258c1287f89d26c6"
server_key_iv: "23cdf77016e05de1a67e5c83379b977a"
server_key_version: "kuduclusterkey@0"
I20260812 01:20:40.094843 10659 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:20:40.105017 10659 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:20:40.107147 10659 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:20:40.107322 10659 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:20:40.112504 10659 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.190:34847
I20260812 01:20:40.112529 10775 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.190:34847 every 8 connection(s)
I20260812 01:20:40.113008 10659 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
I20260812 01:20:40.113747 10776 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:20:40.116012 10776 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db2d2452aea54ae0b06a10744aa3e057: Bootstrap starting.
I20260812 01:20:40.116657 10776 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P db2d2452aea54ae0b06a10744aa3e057: Neither blocks nor log segments found. Creating new log.
I20260812 01:20:40.116966 10776 log.cc:824] T 00000000000000000000000000000000 P db2d2452aea54ae0b06a10744aa3e057: Log is configured to *not* fsync() on all Append() calls
I20260812 01:20:40.117677 10776 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db2d2452aea54ae0b06a10744aa3e057: No bootstrap required, opened a new log
I20260812 01:20:40.119025 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 10659
I20260812 01:20:40.119138 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/minicluster-data/master-0/wal/instance
I20260812 01:20:40.119335 24578 external_mini_cluster.cc:1428] Setting key 9dd81612f95a5e1b0fa638add2b70cec
I20260812 01:20:40.120155 10776 raft_consensus.cc:348] T 00000000000000000000000000000000 P db2d2452aea54ae0b06a10744aa3e057 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db2d2452aea54ae0b06a10744aa3e057" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 34847 } }
I20260812 01:20:40.120360 10776 raft_consensus.cc:374] T 00000000000000000000000000000000 P db2d2452aea54ae0b06a10744aa3e057 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:20:40.120410 10776 raft_consensus.cc:729] T 00000000000000000000000000000000 P db2d2452aea54ae0b06a10744aa3e057 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db2d2452aea54ae0b06a10744aa3e057, State: Initialized, Role: FOLLOWER
I20260812 01:20:40.120569 10776 consensus_queue.cc:260] T 00000000000000000000000000000000 P db2d2452aea54ae0b06a10744aa3e057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db2d2452aea54ae0b06a10744aa3e057" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 34847 } }
I20260812 01:20:40.120672 10776 raft_consensus.cc:388] T 00000000000000000000000000000000 P db2d2452aea54ae0b06a10744aa3e057 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:20:40.120712 10776 raft_consensus.cc:482] T 00000000000000000000000000000000 P db2d2452aea54ae0b06a10744aa3e057 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:20:40.120754 10776 raft_consensus.cc:3037] T 00000000000000000000000000000000 P db2d2452aea54ae0b06a10744aa3e057 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:20:40.121556 10776 raft_consensus.cc:504] T 00000000000000000000000000000000 P db2d2452aea54ae0b06a10744aa3e057 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db2d2452aea54ae0b06a10744aa3e057" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 34847 } }
I20260812 01:20:40.121677 10776 leader_election.cc:302] T 00000000000000000000000000000000 P db2d2452aea54ae0b06a10744aa3e057 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: db2d2452aea54ae0b06a10744aa3e057; no voters:
I20260812 01:20:40.121883 10776 leader_election.cc:288] T 00000000000000000000000000000000 P db2d2452aea54ae0b06a10744aa3e057 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 01:20:40.121914 10789 raft_consensus.cc:2781] T 00000000000000000000000000000000 P db2d2452aea54ae0b06a10744aa3e057 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:20:40.122149 10789 raft_consensus.cc:686] T 00000000000000000000000000000000 P db2d2452aea54ae0b06a10744aa3e057 [term 1 LEADER]: Becoming Leader. State: Replica: db2d2452aea54ae0b06a10744aa3e057, State: Running, Role: LEADER
I20260812 01:20:40.122175 10776 sys_catalog.cc:564] T 00000000000000000000000000000000 P db2d2452aea54ae0b06a10744aa3e057 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:20:40.122888 10789 consensus_queue.cc:237] T 00000000000000000000000000000000 P db2d2452aea54ae0b06a10744aa3e057 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db2d2452aea54ae0b06a10744aa3e057" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 34847 } }
I20260812 01:20:40.123769 10793 sys_catalog.cc:455] T 00000000000000000000000000000000 P db2d2452aea54ae0b06a10744aa3e057 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "db2d2452aea54ae0b06a10744aa3e057" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db2d2452aea54ae0b06a10744aa3e057" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 34847 } } }
I20260812 01:20:40.123883 10793 sys_catalog.cc:458] T 00000000000000000000000000000000 P db2d2452aea54ae0b06a10744aa3e057 [sys.catalog]: This master's current role is: LEADER
I20260812 01:20:40.123983 10776 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260812 01:20:40.124239 10790 sys_catalog.cc:455] T 00000000000000000000000000000000 P db2d2452aea54ae0b06a10744aa3e057 [sys.catalog]: SysCatalogTable state changed. Reason: New leader db2d2452aea54ae0b06a10744aa3e057. Latest consensus state: current_term: 1 leader_uuid: "db2d2452aea54ae0b06a10744aa3e057" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db2d2452aea54ae0b06a10744aa3e057" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 34847 } } }
I20260812 01:20:40.124320 10790 sys_catalog.cc:458] T 00000000000000000000000000000000 P db2d2452aea54ae0b06a10744aa3e057 [sys.catalog]: This master's current role is: LEADER
I20260812 01:20:40.124549 10809 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:20:40.125085 10809 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 12 01:20:40 dist-test-slave-t3q3 krb5kdc[9809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497639, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
I20260812 01:20:40.131213 10812 mvcc.cc:205] Tried to move back new op lower bound from 7317494333971611648 to 7317494333944127488. Current Snapshot: MvccSnapshot[applied={T|T < 7317494333971611648}]
I20260812 01:20:40.134290 10787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:20:40.119957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57094 (local address 127.24.0.190:34847)
0812 01:20:40.120237 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0812 01:20:40.120247 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 01:20:40.120300 (+ 53us) server_negotiation.cc:407] Connection header received
0812 01:20:40.120434 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:20:40.120441 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:20:40.120498 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:20:40.120657 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:20:40.121624 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:20:40.123081 (+ 1457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:20:40.124249 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:20:40.124881 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:20:40.128294 (+ 3413us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:20:40.128922 (+ 628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:20:40.128927 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:20:40.128927 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:20:40.128942 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:20:40.131963 (+ 3021us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:20:40.132438 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:20:40.132441 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:20:40.132442 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:20:40.132595 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:20:40.132828 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:20:40.132829 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:20:40.132829 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:20:40.133211 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:20:40.133821 (+ 610us) server_negotiation.cc:1035] Waiting for connection context
0812 01:20:40.133895 (+ 74us) server_negotiation.cc:299] Negotiation successful
0812 01:20:40.134046 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":231,"thread_start_us":166,"threads_started":1}
I20260812 01:20:40.136556 10809 catalog_manager.cc:1347] Generated new cluster ID: aa71f8e0e73743da8ce88614557981b8
I20260812 01:20:40.136613 10809 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 01:20:40.157514 10809 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:20:40.158349 10809 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:20:40.162150 10809 catalog_manager.cc:5868] T 00000000000000000000000000000000 P db2d2452aea54ae0b06a10744aa3e057: Generated new TSK 0
I20260812 01:20:40.162320 10809 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.0.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.0.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.0.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.129.keytab.
Entry for principal kudu/127.24.0.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.129.keytab.
I20260812 01:20:40.829156 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.0.148:33709/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.0.129:0
--local_ip_for_outbound_sockets=127.24.0.129
--webserver_interface=127.24.0.129
--webserver_port=0
--tserver_master_addrs=127.24.0.190:34847
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.129.keytab
--principal=kudu/127.24.0.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:44139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:20:40.953645 10852 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:20:40.953926 10852 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:20:40.953967 10852 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:20:40.955686 10852 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:20:40.955750 10852 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:20:40.955792 10852 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.0.129
I20260812 01:20:40.957496 10852 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:44139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.24.0.148:33709/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.129.keytab
--principal=kudu/127.24.0.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.0.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.0.129
--webserver_port=0
--tserver_master_addrs=127.24.0.190:34847
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10852
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.0.129
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:20:40.957780 10852 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:20:40.958027 10852 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:20:40.964112 10862 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:20:40.964156 10861 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:20:40.964340 10865 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:20:40.964896 10852 server_base.cc:1034] running on GCE node
I20260812 01:20:40.965111 10852 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:20:40.965378 10852 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:20:40.966554 10852 hybrid_clock.cc:648] HybridClock initialized: now 1786497640966528 us; error 38 us; skew 500 ppm
Aug 12 01:20:40 dist-test-slave-t3q3 krb5kdc[9809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497640, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:20:40.969908 10852 init.cc:377] Logged in from keytab as kudu/127.24.0.129@KRBTEST.COM (short username kudu)
I20260812 01:20:40.970425 10852 webserver.cc:460] Webserver started at http://127.24.0.129:40213/ using document root <none> and password file <none>
I20260812 01:20:40.970728 10852 fs_manager.cc:359] Metadata directory not provided
I20260812 01:20:40.970795 10852 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:20:40.970949 10852 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 01:20:40 dist-test-slave-t3q3 krb5kdc[9809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497640, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.129@KRBTEST.COM for HTTP/127.24.0.148@KRBTEST.COM
I20260812 01:20:40.986817 10852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/minicluster-data/ts-0/data/instance:
uuid: "986e12bf757a465e8bdc59fae37fed20"
format_stamp: "Formatted at 2026-08-12 01:20:40 on dist-test-slave-t3q3"
server_key: "933ed962ad8f995d9401cdfd7d524936"
server_key_iv: "39568c9afb8c40e2145cffeda2f76543"
server_key_version: "kuduclusterkey@0"
I20260812 01:20:40.987373 10852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/minicluster-data/ts-0/wal/instance:
uuid: "986e12bf757a465e8bdc59fae37fed20"
format_stamp: "Formatted at 2026-08-12 01:20:40 on dist-test-slave-t3q3"
server_key: "933ed962ad8f995d9401cdfd7d524936"
server_key_iv: "39568c9afb8c40e2145cffeda2f76543"
server_key_version: "kuduclusterkey@0"
I20260812 01:20:41.004654 10852 fs_manager.cc:691] Time spent creating directory manager: real 0.017s user 0.002s sys 0.000s
I20260812 01:20:41.025136 10876 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:20:41.025352 10852 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:20:41.025476 10852 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/minicluster-data/ts-0/wal
uuid: "986e12bf757a465e8bdc59fae37fed20"
format_stamp: "Formatted at 2026-08-12 01:20:40 on dist-test-slave-t3q3"
server_key: "933ed962ad8f995d9401cdfd7d524936"
server_key_iv: "39568c9afb8c40e2145cffeda2f76543"
server_key_version: "kuduclusterkey@0"
I20260812 01:20:41.025584 10852 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:20:41.054526 10852 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:20:41.056738 10852 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:20:41.056912 10852 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:20:41.057190 10852 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:20:41.057576 10852 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:20:41.057634 10852 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:20:41.057678 10852 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:20:41.057704 10852 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:20:41.065618 10852 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.129:44407
I20260812 01:20:41.065750 11062 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.129:44407 every 8 connection(s)
I20260812 01:20:41.066342 10852 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/minicluster-data/ts-0/data/info.pb
I20260812 01:20:41.069733 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 10852
I20260812 01:20:41.069844 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/minicluster-data/ts-0/wal/instance
I20260812 01:20:41.070034 24578 external_mini_cluster.cc:1428] Setting key b914f34887a5b377be2be7d75778631c
Aug 12 01:20:41 dist-test-slave-t3q3 krb5kdc[9809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497640, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.129@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
I20260812 01:20:41.078667 11066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:20:41.067168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:34847 (local address 127.24.0.129:42953)
0812 01:20:41.067389 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0812 01:20:41.067438 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 01:20:41.068105 (+ 667us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:20:41.068440 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:20:41.068444 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:20:41.068701 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:20:41.068975 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:20:41.068981 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:20:41.070040 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:20:41.070041 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:20:41.071561 (+ 1520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:20:41.071565 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:20:41.072141 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:20:41.072142 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:20:41.072300 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:20:41.072974 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:20:41.073012 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:20:41.074732 (+ 1720us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:20:41.076923 (+ 2191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:20:41.076930 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:20:41.076932 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:20:41.077242 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:20:41.077477 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:20:41.077477 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:20:41.077478 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:20:41.077612 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:20:41.077913 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:20:41.077922 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:20:41.078118 (+ 196us) client_negotiation.cc:769] Sending connection context
0812 01:20:41.078237 (+ 119us) client_negotiation.cc:240] Negotiation successful
0812 01:20:41.078413 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":87,"threads_started":1}
I20260812 01:20:41.079182 11067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:20:41.067217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.129:42953 (local address 127.24.0.190:34847)
0812 01:20:41.067473 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0812 01:20:41.067476 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:20:41.067520 (+ 44us) server_negotiation.cc:407] Connection header received
0812 01:20:41.068232 (+ 712us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:20:41.068234 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:20:41.068250 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:20:41.068351 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:20:41.069091 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:20:41.069909 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:20:41.071674 (+ 1765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:20:41.071956 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:20:41.072042 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:20:41.074848 (+ 2806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:20:41.074849 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:20:41.074850 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:20:41.074858 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:20:41.076818 (+ 1960us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:20:41.077333 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:20:41.077334 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:20:41.077334 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:20:41.077403 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:20:41.077687 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:20:41.077688 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:20:41.077689 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:20:41.077823 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:20:41.077884 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 01:20:41.078979 (+ 1095us) server_negotiation.cc:299] Negotiation successful
0812 01:20:41.079083 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":218,"thread_start_us":88,"threads_started":1}
I20260812 01:20:41.079785 11063 heartbeater.cc:347] Connected to a master server at 127.24.0.190:34847
I20260812 01:20:41.079921 11063 heartbeater.cc:464] Registering TS with master...
I20260812 01:20:41.080169 11063 heartbeater.cc:511] Master 127.24.0.190:34847 requested a full tablet report, sending...
I20260812 01:20:41.080739 10713 ts_manager.cc:194] Registered new tserver with Master: 986e12bf757a465e8bdc59fae37fed20 (127.24.0.129:44407)
WARNING: no policy specified for kudu/127.24.0.130@KRBTEST.COM; defaulting to no policy
I20260812 01:20:41.081585 10713 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.0.129@KRBTEST.COM'} at 127.24.0.129:42953
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.0.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.0.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.130.keytab.
Entry for principal kudu/127.24.0.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.130.keytab.
I20260812 01:20:41.099182 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.0.148:33709/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.0.130:0
--local_ip_for_outbound_sockets=127.24.0.130
--webserver_interface=127.24.0.130
--webserver_port=0
--tserver_master_addrs=127.24.0.190:34847
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.130.keytab
--principal=kudu/127.24.0.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:44139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 01:20:41 dist-test-slave-t3q3 krb5kdc[9809](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786497641, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260812 01:20:41.225600 11071 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:20:41.225971 11071 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:20:41.226063 11071 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:20:41.227921 11071 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:20:41.228039 11071 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:20:41.228165 11071 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.0.130
I20260812 01:20:41.230471 11071 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:44139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.24.0.148:33709/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.130.keytab
--principal=kudu/127.24.0.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.0.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.0.130
--webserver_port=0
--tserver_master_addrs=127.24.0.190:34847
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11071
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.0.130
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:20:41.230926 11071 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:20:41.231256 11071 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:20:41.237907 11096 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:20:41.237923 11094 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:20:41.238168 11099 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:20:41.238466 11071 server_base.cc:1034] running on GCE node
I20260812 01:20:41.238689 11071 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:20:41.238965 11071 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:20:41.240145 11071 hybrid_clock.cc:648] HybridClock initialized: now 1786497641240126 us; error 35 us; skew 500 ppm
Aug 12 01:20:41 dist-test-slave-t3q3 krb5kdc[9809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497641, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:20:41.243517 11071 init.cc:377] Logged in from keytab as kudu/127.24.0.130@KRBTEST.COM (short username kudu)
I20260812 01:20:41.244115 11071 webserver.cc:460] Webserver started at http://127.24.0.130:45715/ using document root <none> and password file <none>
I20260812 01:20:41.244432 11071 fs_manager.cc:359] Metadata directory not provided
I20260812 01:20:41.244541 11071 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:20:41.244725 11071 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 01:20:41 dist-test-slave-t3q3 krb5kdc[9809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497641, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.130@KRBTEST.COM for HTTP/127.24.0.148@KRBTEST.COM
I20260812 01:20:41.264948 11071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/minicluster-data/ts-1/data/instance:
uuid: "6b2426cb13c543638ca6a03934d5b5eb"
format_stamp: "Formatted at 2026-08-12 01:20:41 on dist-test-slave-t3q3"
server_key: "a4790582329b5a7890923797ea4c5e14"
server_key_iv: "7afbb8615c95b4ce39fc5e3002c3a687"
server_key_version: "kuduclusterkey@0"
I20260812 01:20:41.265520 11071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/minicluster-data/ts-1/wal/instance:
uuid: "6b2426cb13c543638ca6a03934d5b5eb"
format_stamp: "Formatted at 2026-08-12 01:20:41 on dist-test-slave-t3q3"
server_key: "a4790582329b5a7890923797ea4c5e14"
server_key_iv: "7afbb8615c95b4ce39fc5e3002c3a687"
server_key_version: "kuduclusterkey@0"
I20260812 01:20:41.267355 11071 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260812 01:20:41.292479 11111 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:20:41.292804 11071 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.000s sys 0.000s
I20260812 01:20:41.292934 11071 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/minicluster-data/ts-1/wal
uuid: "6b2426cb13c543638ca6a03934d5b5eb"
format_stamp: "Formatted at 2026-08-12 01:20:41 on dist-test-slave-t3q3"
server_key: "a4790582329b5a7890923797ea4c5e14"
server_key_iv: "7afbb8615c95b4ce39fc5e3002c3a687"
server_key_version: "kuduclusterkey@0"
I20260812 01:20:41.293064 11071 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:20:41.327805 11071 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:20:41.330497 11071 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:20:41.330678 11071 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:20:41.331022 11071 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:20:41.331450 11071 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:20:41.331507 11071 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:20:41.331542 11071 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:20:41.331625 11071 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:20:41.338595 11071 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.130:34575
I20260812 01:20:41.339156 11071 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/minicluster-data/ts-1/data/info.pb
I20260812 01:20:41.347404 11318 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.130:34575 every 8 connection(s)
I20260812 01:20:41.348026 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 11071
I20260812 01:20:41.348170 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/minicluster-data/ts-1/wal/instance
I20260812 01:20:41.348397 24578 external_mini_cluster.cc:1428] Setting key 8e532fa818b17052bab81dbdc066743e
WARNING: no policy specified for kudu/127.24.0.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.0.131@KRBTEST.COM" created.
Aug 12 01:20:41 dist-test-slave-t3q3 krb5kdc[9809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497641, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.130@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
I20260812 01:20:41.371199 11067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:20:41.347056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.130:50665 (local address 127.24.0.190:34847)
0812 01:20:41.347140 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0812 01:20:41.347143 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:20:41.348486 (+ 1343us) server_negotiation.cc:407] Connection header received
0812 01:20:41.349200 (+ 714us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:20:41.349201 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:20:41.349218 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:20:41.349261 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:20:41.350023 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:20:41.350832 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:20:41.354613 (+ 3781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:20:41.354936 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:20:41.354986 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:20:41.365720 (+ 10734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:20:41.365722 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:20:41.365723 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:20:41.365733 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:20:41.367610 (+ 1877us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:20:41.369844 (+ 2234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:20:41.369845 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:20:41.369846 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:20:41.369919 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:20:41.370298 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:20:41.370299 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:20:41.370300 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:20:41.370464 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:20:41.370529 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 01:20:41.370962 (+ 433us) server_negotiation.cc:299] Negotiation successful
0812 01:20:41.371090 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260812 01:20:41.372128 11325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:20:41.347050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:34847 (local address 127.24.0.130:50665)
0812 01:20:41.348338 (+ 1288us) negotiation.cc:107] Waiting for socket to connect
0812 01:20:41.348390 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 01:20:41.349088 (+ 698us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:20:41.349345 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:20:41.349350 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:20:41.349633 (+ 283us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:20:41.349929 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:20:41.349935 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:20:41.353181 (+ 3246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:20:41.353183 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:20:41.354496 (+ 1313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:20:41.354500 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:20:41.355087 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:20:41.355088 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:20:41.355257 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:20:41.363660 (+ 8403us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:20:41.363673 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:20:41.365540 (+ 1867us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:20:41.369331 (+ 3791us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:20:41.369342 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:20:41.369346 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:20:41.369721 (+ 375us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:20:41.370012 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:20:41.370013 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:20:41.370013 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:20:41.370197 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:20:41.370572 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:20:41.370580 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:20:41.370779 (+ 199us) client_negotiation.cc:769] Sending connection context
0812 01:20:41.371618 (+ 839us) client_negotiation.cc:240] Negotiation successful
0812 01:20:41.371818 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1235,"thread_start_us":125,"threads_started":1}
I20260812 01:20:41.379016 11319 heartbeater.cc:347] Connected to a master server at 127.24.0.190:34847
I20260812 01:20:41.379153 11319 heartbeater.cc:464] Registering TS with master...
I20260812 01:20:41.379391 11319 heartbeater.cc:511] Master 127.24.0.190:34847 requested a full tablet report, sending...
I20260812 01:20:41.379923 10713 ts_manager.cc:194] Registered new tserver with Master: 6b2426cb13c543638ca6a03934d5b5eb (127.24.0.130:34575)
I20260812 01:20:41.380488 10713 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.0.130@KRBTEST.COM'} at 127.24.0.130:50665
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.0.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.131.keytab.
Entry for principal kudu/127.24.0.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.131.keytab.
I20260812 01:20:41.389420 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.0.148:33709/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.0.131:0
--local_ip_for_outbound_sockets=127.24.0.131
--webserver_interface=127.24.0.131
--webserver_port=0
--tserver_master_addrs=127.24.0.190:34847
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.131.keytab
--principal=kudu/127.24.0.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:44139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:20:41.529204 11333 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:20:41.529699 11333 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:20:41.529804 11333 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:20:41.531795 11333 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:20:41.531917 11333 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:20:41.532021 11333 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.0.131
I20260812 01:20:41.534433 11333 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:44139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.24.0.148:33709/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.131.keytab
--principal=kudu/127.24.0.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.0.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.0.131
--webserver_port=0
--tserver_master_addrs=127.24.0.190:34847
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11333
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.0.131
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:20:41.534972 11333 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:20:41.535352 11333 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:20:41.542840 11344 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:20:41.542865 11347 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:20:41.542866 11345 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:20:41.543965 11333 server_base.cc:1034] running on GCE node
I20260812 01:20:41.544255 11333 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:20:41.544628 11333 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:20:41.545818 11333 hybrid_clock.cc:648] HybridClock initialized: now 1786497641545791 us; error 41 us; skew 500 ppm
Aug 12 01:20:41 dist-test-slave-t3q3 krb5kdc[9809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497641, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:20:41.549463 11333 init.cc:377] Logged in from keytab as kudu/127.24.0.131@KRBTEST.COM (short username kudu)
I20260812 01:20:41.550057 11333 webserver.cc:460] Webserver started at http://127.24.0.131:34545/ using document root <none> and password file <none>
I20260812 01:20:41.550348 11333 fs_manager.cc:359] Metadata directory not provided
I20260812 01:20:41.550402 11333 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:20:41.550549 11333 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 01:20:41 dist-test-slave-t3q3 krb5kdc[9809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497641, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.131@KRBTEST.COM for HTTP/127.24.0.148@KRBTEST.COM
I20260812 01:20:41.575843 11333 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/minicluster-data/ts-2/data/instance:
uuid: "945bd684dd674b49802e4a23cc6ac7fc"
format_stamp: "Formatted at 2026-08-12 01:20:41 on dist-test-slave-t3q3"
server_key: "2e6e3af0fe79a1ab5859a98188766a9a"
server_key_iv: "fc4d8a843c6ca9967c687b353cefe966"
server_key_version: "kuduclusterkey@0"
I20260812 01:20:41.576539 11333 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/minicluster-data/ts-2/wal/instance:
uuid: "945bd684dd674b49802e4a23cc6ac7fc"
format_stamp: "Formatted at 2026-08-12 01:20:41 on dist-test-slave-t3q3"
server_key: "2e6e3af0fe79a1ab5859a98188766a9a"
server_key_iv: "fc4d8a843c6ca9967c687b353cefe966"
server_key_version: "kuduclusterkey@0"
I20260812 01:20:41.584496 11333 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.001s sys 0.000s
I20260812 01:20:41.608855 11358 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:20:41.609177 11333 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.001s sys 0.000s
I20260812 01:20:41.609295 11333 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/minicluster-data/ts-2/wal
uuid: "945bd684dd674b49802e4a23cc6ac7fc"
format_stamp: "Formatted at 2026-08-12 01:20:41 on dist-test-slave-t3q3"
server_key: "2e6e3af0fe79a1ab5859a98188766a9a"
server_key_iv: "fc4d8a843c6ca9967c687b353cefe966"
server_key_version: "kuduclusterkey@0"
I20260812 01:20:41.609407 11333 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:20:41.630508 11333 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:20:41.633721 11333 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:20:41.633939 11333 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:20:41.634326 11333 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:20:41.634789 11333 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:20:41.634883 11333 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:20:41.634956 11333 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:20:41.635016 11333 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:20:41.643486 11333 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.131:36545
I20260812 01:20:41.643988 11333 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/minicluster-data/ts-2/data/info.pb
I20260812 01:20:41.646641 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 11333
I20260812 01:20:41.646752 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/minicluster-data/ts-2/wal/instance
I20260812 01:20:41.646929 24578 external_mini_cluster.cc:1428] Setting key 044410dad4538b81727383aba25c40b0
I20260812 01:20:41.656600 11541 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.131:36545 every 8 connection(s)
Aug 12 01:20:41 dist-test-slave-t3q3 krb5kdc[9809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497641, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.131@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
I20260812 01:20:41.668641 11067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:20:41.657681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.131:33465 (local address 127.24.0.190:34847)
0812 01:20:41.657728 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 01:20:41.657730 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:20:41.658075 (+ 345us) server_negotiation.cc:407] Connection header received
0812 01:20:41.658786 (+ 711us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:20:41.658787 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:20:41.658799 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:20:41.658831 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:20:41.659717 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:20:41.660500 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:20:41.662047 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:20:41.662347 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:20:41.662389 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:20:41.665211 (+ 2822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:20:41.665213 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:20:41.665213 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:20:41.665221 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:20:41.666948 (+ 1727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:20:41.667445 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:20:41.667446 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:20:41.667446 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:20:41.667509 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:20:41.667819 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:20:41.667820 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:20:41.667821 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:20:41.667959 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:20:41.668006 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0812 01:20:41.668447 (+ 441us) server_negotiation.cc:299] Negotiation successful
0812 01:20:41.668547 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 01:20:41.669138 11547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:20:41.657273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:34847 (local address 127.24.0.131:33465)
0812 01:20:41.657935 (+ 662us) negotiation.cc:107] Waiting for socket to connect
0812 01:20:41.657982 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 01:20:41.658655 (+ 673us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:20:41.659024 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:20:41.659028 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:20:41.659289 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:20:41.659605 (+ 316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:20:41.659611 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:20:41.660639 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:20:41.660640 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:20:41.661936 (+ 1296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:20:41.661939 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:20:41.662483 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:20:41.662484 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:20:41.662642 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:20:41.663385 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:20:41.663396 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:20:41.665087 (+ 1691us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:20:41.667047 (+ 1960us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:20:41.667054 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:20:41.667056 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:20:41.667349 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:20:41.667603 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:20:41.667604 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:20:41.667604 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:20:41.667738 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:20:41.668100 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:20:41.668107 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:20:41.668273 (+ 166us) client_negotiation.cc:769] Sending connection context
0812 01:20:41.668741 (+ 468us) client_negotiation.cc:240] Negotiation successful
0812 01:20:41.668906 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":621,"thread_start_us":194,"threads_started":1}
I20260812 01:20:41.669715 11542 heartbeater.cc:347] Connected to a master server at 127.24.0.190:34847
I20260812 01:20:41.669812 11542 heartbeater.cc:464] Registering TS with master...
I20260812 01:20:41.669996 11542 heartbeater.cc:511] Master 127.24.0.190:34847 requested a full tablet report, sending...
I20260812 01:20:41.670410 10713 ts_manager.cc:194] Registered new tserver with Master: 945bd684dd674b49802e4a23cc6ac7fc (127.24.0.131:36545)
I20260812 01:20:41.670863 10713 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.0.131@KRBTEST.COM'} at 127.24.0.131:33465
I20260812 01:20:41.680195 24578 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 01:20:41.690650 11067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:20:41.682191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57100 (local address 127.24.0.190:34847)
0812 01:20:41.682258 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0812 01:20:41.682260 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:20:41.682727 (+ 467us) server_negotiation.cc:407] Connection header received
0812 01:20:41.682836 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:20:41.682837 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:20:41.682847 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:20:41.682878 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:20:41.683683 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:20:41.684453 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:20:41.685596 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:20:41.685888 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:20:41.685934 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:20:41.687518 (+ 1584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:20:41.687519 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:20:41.687520 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:20:41.687527 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:20:41.689286 (+ 1759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:20:41.689713 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:20:41.689715 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:20:41.689715 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:20:41.689771 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:20:41.689993 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:20:41.689994 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:20:41.689995 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:20:41.690134 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:20:41.690213 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 01:20:41.690467 (+ 254us) server_negotiation.cc:299] Negotiation successful
0812 01:20:41.690546 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 01:20:41.692745 10713 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57100:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 01:20:41.695170 10713 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 01:20:41.706054 11572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:20:41.698879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.130:34575 (local address 127.0.0.1:35478)
0812 01:20:41.700611 (+ 1732us) negotiation.cc:107] Waiting for socket to connect
0812 01:20:41.700654 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 01:20:41.700723 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:20:41.701243 (+ 520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:20:41.701247 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:20:41.701249 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:20:41.701322 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:20:41.701324 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:20:41.703028 (+ 1704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:20:41.703029 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:20:41.704734 (+ 1705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:20:41.704737 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:20:41.705768 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:20:41.705769 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:20:41.705849 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:20:41.705865 (+ 16us) client_negotiation.cc:769] Sending connection context
0812 01:20:41.705919 (+ 54us) client_negotiation.cc:240] Negotiation successful
0812 01:20:41.705970 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1686,"thread_start_us":124,"threads_started":1}
I20260812 01:20:41.706621 11570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:20:41.698995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35478 (local address 127.24.0.130:34575)
0812 01:20:41.700995 (+ 2000us) server_negotiation.cc:206] Beginning negotiation
0812 01:20:41.701000 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 01:20:41.701010 (+ 10us) server_negotiation.cc:407] Connection header received
0812 01:20:41.701053 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:20:41.701055 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:20:41.701071 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:20:41.701150 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:20:41.701412 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:20:41.702923 (+ 1511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:20:41.704866 (+ 1943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:20:41.705658 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:20:41.706287 (+ 629us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:20:41.706377 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 01:20:41.706451 (+ 74us) server_negotiation.cc:299] Negotiation successful
0812 01:20:41.706527 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1964,"thread_start_us":103,"threads_started":1}
I20260812 01:20:41.707371 11188 tablet_service.cc:1467] Processing CreateTablet for tablet 55e7f171b30a42eb9e4dee601edf46da (DEFAULT_TABLE table=test-table [id=348b06b3bca643f59e5029c626d21f2e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:20:41.707801 11188 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55e7f171b30a42eb9e4dee601edf46da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:20:41.712175 11583 tablet_bootstrap.cc:492] T 55e7f171b30a42eb9e4dee601edf46da P 6b2426cb13c543638ca6a03934d5b5eb: Bootstrap starting.
I20260812 01:20:41.713274 11583 tablet_bootstrap.cc:654] T 55e7f171b30a42eb9e4dee601edf46da P 6b2426cb13c543638ca6a03934d5b5eb: Neither blocks nor log segments found. Creating new log.
I20260812 01:20:41.713750 11583 log.cc:824] T 55e7f171b30a42eb9e4dee601edf46da P 6b2426cb13c543638ca6a03934d5b5eb: Log is configured to *not* fsync() on all Append() calls
I20260812 01:20:41.714905 11583 tablet_bootstrap.cc:492] T 55e7f171b30a42eb9e4dee601edf46da P 6b2426cb13c543638ca6a03934d5b5eb: No bootstrap required, opened a new log
I20260812 01:20:41.714980 11583 ts_tablet_manager.cc:1397] T 55e7f171b30a42eb9e4dee601edf46da P 6b2426cb13c543638ca6a03934d5b5eb: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260812 01:20:41.717094 11571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:20:41.699181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.129:44407 (local address 127.0.0.1:45600)
0812 01:20:41.700832 (+ 1651us) negotiation.cc:107] Waiting for socket to connect
0812 01:20:41.700841 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 01:20:41.700878 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:20:41.704551 (+ 3673us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:20:41.704552 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:20:41.704553 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:20:41.704603 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:20:41.704605 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:20:41.712579 (+ 7974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:20:41.712580 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:20:41.713666 (+ 1086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:20:41.713670 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:20:41.716904 (+ 3234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:20:41.716905 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:20:41.716957 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:20:41.716961 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 01:20:41.716986 (+ 25us) client_negotiation.cc:240] Negotiation successful
0812 01:20:41.717022 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1638,"thread_start_us":80,"threads_started":1}
I20260812 01:20:41.716990 11583 raft_consensus.cc:348] T 55e7f171b30a42eb9e4dee601edf46da P 6b2426cb13c543638ca6a03934d5b5eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "986e12bf757a465e8bdc59fae37fed20" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 44407 } } peers { permanent_uuid: "6b2426cb13c543638ca6a03934d5b5eb" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 34575 } } peers { permanent_uuid: "945bd684dd674b49802e4a23cc6ac7fc" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 36545 } }
I20260812 01:20:41.717165 11583 raft_consensus.cc:374] T 55e7f171b30a42eb9e4dee601edf46da P 6b2426cb13c543638ca6a03934d5b5eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:20:41.717206 11583 raft_consensus.cc:729] T 55e7f171b30a42eb9e4dee601edf46da P 6b2426cb13c543638ca6a03934d5b5eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b2426cb13c543638ca6a03934d5b5eb, State: Initialized, Role: FOLLOWER
I20260812 01:20:41.717334 11583 consensus_queue.cc:260] T 55e7f171b30a42eb9e4dee601edf46da P 6b2426cb13c543638ca6a03934d5b5eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "986e12bf757a465e8bdc59fae37fed20" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 44407 } } peers { permanent_uuid: "6b2426cb13c543638ca6a03934d5b5eb" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 34575 } } peers { permanent_uuid: "945bd684dd674b49802e4a23cc6ac7fc" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 36545 } }
I20260812 01:20:41.717518 11575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:20:41.699827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45600 (local address 127.24.0.129:44407)
0812 01:20:41.704198 (+ 4371us) server_negotiation.cc:206] Beginning negotiation
0812 01:20:41.704207 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 01:20:41.704225 (+ 18us) server_negotiation.cc:407] Connection header received
0812 01:20:41.704264 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:20:41.704266 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:20:41.704283 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:20:41.704678 (+ 395us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:20:41.704741 (+ 63us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:20:41.705839 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:20:41.713771 (+ 7932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:20:41.714501 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:20:41.717288 (+ 2787us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:20:41.717364 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 01:20:41.717393 (+ 29us) server_negotiation.cc:299] Negotiation successful
0812 01:20:41.717439 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4330,"thread_start_us":59,"threads_started":1}
I20260812 01:20:41.717651 11583 ts_tablet_manager.cc:1428] T 55e7f171b30a42eb9e4dee601edf46da P 6b2426cb13c543638ca6a03934d5b5eb: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 01:20:41.718101 11319 heartbeater.cc:503] Master 127.24.0.190:34847 was elected leader, sending a full tablet report...
I20260812 01:20:41.718135 10961 tablet_service.cc:1467] Processing CreateTablet for tablet 55e7f171b30a42eb9e4dee601edf46da (DEFAULT_TABLE table=test-table [id=348b06b3bca643f59e5029c626d21f2e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:20:41.718484 10961 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55e7f171b30a42eb9e4dee601edf46da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:20:41.721854 11569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:20:41.699348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.131:36545 (local address 127.0.0.1:36100)
0812 01:20:41.704408 (+ 5060us) negotiation.cc:107] Waiting for socket to connect
0812 01:20:41.704418 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 01:20:41.704460 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:20:41.712424 (+ 7964us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:20:41.712425 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:20:41.712427 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:20:41.712467 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:20:41.712469 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:20:41.715721 (+ 3252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:20:41.715722 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:20:41.716789 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:20:41.716792 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:20:41.721640 (+ 4848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:20:41.721641 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:20:41.721702 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:20:41.721705 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 01:20:41.721756 (+ 51us) client_negotiation.cc:240] Negotiation successful
0812 01:20:41.721792 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5042,"thread_start_us":199,"threads_started":1}
I20260812 01:20:41.722328 11577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:20:41.699976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36100 (local address 127.24.0.131:36545)
0812 01:20:41.712180 (+ 12204us) server_negotiation.cc:206] Beginning negotiation
0812 01:20:41.712184 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 01:20:41.712196 (+ 12us) server_negotiation.cc:407] Connection header received
0812 01:20:41.712236 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:20:41.712238 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:20:41.712255 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:20:41.712336 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:20:41.714609 (+ 2273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:20:41.715628 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:20:41.719417 (+ 3789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:20:41.720250 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:20:41.720323 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:20:41.720398 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 01:20:41.722016 (+ 1618us) server_negotiation.cc:299] Negotiation successful
0812 01:20:41.722251 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12158,"thread_start_us":217,"threads_started":1}
I20260812 01:20:41.722981 11412 tablet_service.cc:1467] Processing CreateTablet for tablet 55e7f171b30a42eb9e4dee601edf46da (DEFAULT_TABLE table=test-table [id=348b06b3bca643f59e5029c626d21f2e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:20:41.723304 11412 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55e7f171b30a42eb9e4dee601edf46da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:20:41.726590 11588 tablet_bootstrap.cc:492] T 55e7f171b30a42eb9e4dee601edf46da P 945bd684dd674b49802e4a23cc6ac7fc: Bootstrap starting.
I20260812 01:20:41.727846 11588 tablet_bootstrap.cc:654] T 55e7f171b30a42eb9e4dee601edf46da P 945bd684dd674b49802e4a23cc6ac7fc: Neither blocks nor log segments found. Creating new log.
I20260812 01:20:41.728196 11588 log.cc:824] T 55e7f171b30a42eb9e4dee601edf46da P 945bd684dd674b49802e4a23cc6ac7fc: Log is configured to *not* fsync() on all Append() calls
I20260812 01:20:41.728535 11587 tablet_bootstrap.cc:492] T 55e7f171b30a42eb9e4dee601edf46da P 986e12bf757a465e8bdc59fae37fed20: Bootstrap starting.
I20260812 01:20:41.729241 11587 tablet_bootstrap.cc:654] T 55e7f171b30a42eb9e4dee601edf46da P 986e12bf757a465e8bdc59fae37fed20: Neither blocks nor log segments found. Creating new log.
I20260812 01:20:41.729568 11587 log.cc:824] T 55e7f171b30a42eb9e4dee601edf46da P 986e12bf757a465e8bdc59fae37fed20: Log is configured to *not* fsync() on all Append() calls
I20260812 01:20:41.730348 11588 tablet_bootstrap.cc:492] T 55e7f171b30a42eb9e4dee601edf46da P 945bd684dd674b49802e4a23cc6ac7fc: No bootstrap required, opened a new log
I20260812 01:20:41.730430 11588 ts_tablet_manager.cc:1397] T 55e7f171b30a42eb9e4dee601edf46da P 945bd684dd674b49802e4a23cc6ac7fc: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260812 01:20:41.732435 11588 raft_consensus.cc:348] T 55e7f171b30a42eb9e4dee601edf46da P 945bd684dd674b49802e4a23cc6ac7fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "986e12bf757a465e8bdc59fae37fed20" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 44407 } } peers { permanent_uuid: "6b2426cb13c543638ca6a03934d5b5eb" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 34575 } } peers { permanent_uuid: "945bd684dd674b49802e4a23cc6ac7fc" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 36545 } }
I20260812 01:20:41.732620 11588 raft_consensus.cc:374] T 55e7f171b30a42eb9e4dee601edf46da P 945bd684dd674b49802e4a23cc6ac7fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:20:41.732667 11588 raft_consensus.cc:729] T 55e7f171b30a42eb9e4dee601edf46da P 945bd684dd674b49802e4a23cc6ac7fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 945bd684dd674b49802e4a23cc6ac7fc, State: Initialized, Role: FOLLOWER
I20260812 01:20:41.732818 11588 consensus_queue.cc:260] T 55e7f171b30a42eb9e4dee601edf46da P 945bd684dd674b49802e4a23cc6ac7fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "986e12bf757a465e8bdc59fae37fed20" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 44407 } } peers { permanent_uuid: "6b2426cb13c543638ca6a03934d5b5eb" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 34575 } } peers { permanent_uuid: "945bd684dd674b49802e4a23cc6ac7fc" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 36545 } }
I20260812 01:20:41.733110 11588 ts_tablet_manager.cc:1428] T 55e7f171b30a42eb9e4dee601edf46da P 945bd684dd674b49802e4a23cc6ac7fc: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 01:20:41.733318 11542 heartbeater.cc:503] Master 127.24.0.190:34847 was elected leader, sending a full tablet report...
I20260812 01:20:41.734159 11587 tablet_bootstrap.cc:492] T 55e7f171b30a42eb9e4dee601edf46da P 986e12bf757a465e8bdc59fae37fed20: No bootstrap required, opened a new log
I20260812 01:20:41.734236 11587 ts_tablet_manager.cc:1397] T 55e7f171b30a42eb9e4dee601edf46da P 986e12bf757a465e8bdc59fae37fed20: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260812 01:20:41.736902 11587 raft_consensus.cc:348] T 55e7f171b30a42eb9e4dee601edf46da P 986e12bf757a465e8bdc59fae37fed20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "986e12bf757a465e8bdc59fae37fed20" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 44407 } } peers { permanent_uuid: "6b2426cb13c543638ca6a03934d5b5eb" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 34575 } } peers { permanent_uuid: "945bd684dd674b49802e4a23cc6ac7fc" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 36545 } }
I20260812 01:20:41.738292 11587 raft_consensus.cc:374] T 55e7f171b30a42eb9e4dee601edf46da P 986e12bf757a465e8bdc59fae37fed20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:20:41.738341 11587 raft_consensus.cc:729] T 55e7f171b30a42eb9e4dee601edf46da P 986e12bf757a465e8bdc59fae37fed20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 986e12bf757a465e8bdc59fae37fed20, State: Initialized, Role: FOLLOWER
I20260812 01:20:41.738487 11587 consensus_queue.cc:260] T 55e7f171b30a42eb9e4dee601edf46da P 986e12bf757a465e8bdc59fae37fed20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "986e12bf757a465e8bdc59fae37fed20" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 44407 } } peers { permanent_uuid: "6b2426cb13c543638ca6a03934d5b5eb" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 34575 } } peers { permanent_uuid: "945bd684dd674b49802e4a23cc6ac7fc" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 36545 } }
I20260812 01:20:41.738770 11587 ts_tablet_manager.cc:1428] T 55e7f171b30a42eb9e4dee601edf46da P 986e12bf757a465e8bdc59fae37fed20: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260812 01:20:41.738893 11063 heartbeater.cc:503] Master 127.24.0.190:34847 was elected leader, sending a full tablet report...
Aug 12 01:20:41 dist-test-slave-t3q3 krb5kdc[9809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497641, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.190@KRBTEST.COM for HTTP/127.24.0.148@KRBTEST.COM
I20260812 01:20:41.814177 11595 raft_consensus.cc:482] T 55e7f171b30a42eb9e4dee601edf46da P 945bd684dd674b49802e4a23cc6ac7fc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 01:20:41.814352 11595 raft_consensus.cc:504] T 55e7f171b30a42eb9e4dee601edf46da P 945bd684dd674b49802e4a23cc6ac7fc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "986e12bf757a465e8bdc59fae37fed20" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 44407 } } peers { permanent_uuid: "6b2426cb13c543638ca6a03934d5b5eb" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 34575 } } peers { permanent_uuid: "945bd684dd674b49802e4a23cc6ac7fc" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 36545 } }
I20260812 01:20:41.814805 11595 leader_election.cc:288] T 55e7f171b30a42eb9e4dee601edf46da P 945bd684dd674b49802e4a23cc6ac7fc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 986e12bf757a465e8bdc59fae37fed20 (127.24.0.129:44407), 6b2426cb13c543638ca6a03934d5b5eb (127.24.0.130:34575)
W20260812 01:20:41.817857 11065 tablet.cc:2369] T 55e7f171b30a42eb9e4dee601edf46da P 986e12bf757a465e8bdc59fae37fed20: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 01:20:41.819193 11547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:20:41.815065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.130:34575 (local address 127.24.0.131:47883)
0812 01:20:41.815235 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0812 01:20:41.815249 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 01:20:41.815301 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:20:41.816084 (+ 783us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:20:41.816086 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:20:41.816088 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:20:41.816177 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:20:41.816179 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:20:41.817168 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:20:41.817169 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:20:41.818266 (+ 1097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:20:41.818269 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:20:41.818968 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:20:41.818969 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:20:41.819039 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:20:41.819043 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 01:20:41.819086 (+ 43us) client_negotiation.cc:240] Negotiation successful
0812 01:20:41.819124 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145}
I20260812 01:20:41.819681 11570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:20:41.815400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.131:47883 (local address 127.24.0.130:34575)
0812 01:20:41.815452 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 01:20:41.815454 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:20:41.815463 (+ 9us) server_negotiation.cc:407] Connection header received
0812 01:20:41.815496 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:20:41.815497 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:20:41.815523 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:20:41.815578 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:20:41.816276 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:20:41.817045 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:20:41.818361 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:20:41.818864 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:20:41.818928 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:20:41.818937 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 01:20:41.819574 (+ 637us) server_negotiation.cc:299] Negotiation successful
0812 01:20:41.819616 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 01:20:41.819964 11236 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55e7f171b30a42eb9e4dee601edf46da" candidate_uuid: "945bd684dd674b49802e4a23cc6ac7fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b2426cb13c543638ca6a03934d5b5eb" is_pre_election: true
I20260812 01:20:41.820180 11236 raft_consensus.cc:2454] T 55e7f171b30a42eb9e4dee601edf46da P 6b2426cb13c543638ca6a03934d5b5eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 945bd684dd674b49802e4a23cc6ac7fc in term 0.
I20260812 01:20:41.821264 11360 leader_election.cc:302] T 55e7f171b30a42eb9e4dee601edf46da P 945bd684dd674b49802e4a23cc6ac7fc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b2426cb13c543638ca6a03934d5b5eb, 945bd684dd674b49802e4a23cc6ac7fc; no voters:
I20260812 01:20:41.821403 11595 raft_consensus.cc:2781] T 55e7f171b30a42eb9e4dee601edf46da P 945bd684dd674b49802e4a23cc6ac7fc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 01:20:41.821448 11595 raft_consensus.cc:482] T 55e7f171b30a42eb9e4dee601edf46da P 945bd684dd674b49802e4a23cc6ac7fc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 01:20:41.821470 11595 raft_consensus.cc:3037] T 55e7f171b30a42eb9e4dee601edf46da P 945bd684dd674b49802e4a23cc6ac7fc [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:20:41.822372 11595 raft_consensus.cc:504] T 55e7f171b30a42eb9e4dee601edf46da P 945bd684dd674b49802e4a23cc6ac7fc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "986e12bf757a465e8bdc59fae37fed20" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 44407 } } peers { permanent_uuid: "6b2426cb13c543638ca6a03934d5b5eb" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 34575 } } peers { permanent_uuid: "945bd684dd674b49802e4a23cc6ac7fc" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 36545 } }
I20260812 01:20:41.822547 11595 leader_election.cc:288] T 55e7f171b30a42eb9e4dee601edf46da P 945bd684dd674b49802e4a23cc6ac7fc [CANDIDATE]: Term 1 election: Requested vote from peers 986e12bf757a465e8bdc59fae37fed20 (127.24.0.129:44407), 6b2426cb13c543638ca6a03934d5b5eb (127.24.0.130:34575)
I20260812 01:20:41.822809 11236 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55e7f171b30a42eb9e4dee601edf46da" candidate_uuid: "945bd684dd674b49802e4a23cc6ac7fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b2426cb13c543638ca6a03934d5b5eb"
I20260812 01:20:41.822885 11236 raft_consensus.cc:3037] T 55e7f171b30a42eb9e4dee601edf46da P 6b2426cb13c543638ca6a03934d5b5eb [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:20:41.823745 11236 raft_consensus.cc:2454] T 55e7f171b30a42eb9e4dee601edf46da P 6b2426cb13c543638ca6a03934d5b5eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 945bd684dd674b49802e4a23cc6ac7fc in term 1.
I20260812 01:20:41.823926 11360 leader_election.cc:302] T 55e7f171b30a42eb9e4dee601edf46da P 945bd684dd674b49802e4a23cc6ac7fc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b2426cb13c543638ca6a03934d5b5eb, 945bd684dd674b49802e4a23cc6ac7fc; no voters:
I20260812 01:20:41.824002 11595 raft_consensus.cc:2781] T 55e7f171b30a42eb9e4dee601edf46da P 945bd684dd674b49802e4a23cc6ac7fc [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:20:41.825984 11595 raft_consensus.cc:686] T 55e7f171b30a42eb9e4dee601edf46da P 945bd684dd674b49802e4a23cc6ac7fc [term 1 LEADER]: Becoming Leader. State: Replica: 945bd684dd674b49802e4a23cc6ac7fc, State: Running, Role: LEADER
I20260812 01:20:41.826103 11595 consensus_queue.cc:237] T 55e7f171b30a42eb9e4dee601edf46da P 945bd684dd674b49802e4a23cc6ac7fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "986e12bf757a465e8bdc59fae37fed20" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 44407 } } peers { permanent_uuid: "6b2426cb13c543638ca6a03934d5b5eb" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 34575 } } peers { permanent_uuid: "945bd684dd674b49802e4a23cc6ac7fc" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 36545 } }
I20260812 01:20:41.826921 10713 catalog_manager.cc:5495] T 55e7f171b30a42eb9e4dee601edf46da P 945bd684dd674b49802e4a23cc6ac7fc reported cstate change: term changed from 0 to 1, leader changed from <none> to 945bd684dd674b49802e4a23cc6ac7fc (127.24.0.131). New cstate: current_term: 1 leader_uuid: "945bd684dd674b49802e4a23cc6ac7fc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "986e12bf757a465e8bdc59fae37fed20" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 44407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b2426cb13c543638ca6a03934d5b5eb" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 34575 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "945bd684dd674b49802e4a23cc6ac7fc" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 36545 } health_report { overall_health: HEALTHY } } }
I20260812 01:20:41.827847 11575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:20:41.815975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.131:46579 (local address 127.24.0.129:44407)
0812 01:20:41.816018 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 01:20:41.816021 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:20:41.819366 (+ 3345us) server_negotiation.cc:407] Connection header received
0812 01:20:41.819397 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:20:41.819398 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:20:41.819411 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:20:41.819452 (+ 41us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:20:41.819962 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:20:41.820680 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:20:41.825231 (+ 4551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:20:41.825794 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:20:41.825861 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:20:41.825870 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 01:20:41.827721 (+ 1851us) server_negotiation.cc:299] Negotiation successful
0812 01:20:41.827778 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 01:20:41.828173 11547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:20:41.815725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.129:44407 (local address 127.24.0.131:46579)
0812 01:20:41.819266 (+ 3541us) negotiation.cc:107] Waiting for socket to connect
0812 01:20:41.819278 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 01:20:41.819316 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:20:41.819852 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:20:41.819853 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:20:41.819854 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:20:41.819885 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:20:41.819886 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:20:41.824125 (+ 4239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:20:41.824127 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:20:41.825130 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:20:41.825133 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:20:41.827462 (+ 2329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:20:41.827463 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:20:41.827536 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:20:41.827540 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 01:20:41.827592 (+ 52us) client_negotiation.cc:240] Negotiation successful
0812 01:20:41.828110 (+ 518us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3531,"thread_start_us":126,"threads_started":1}
I20260812 01:20:41.828554 10997 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55e7f171b30a42eb9e4dee601edf46da" candidate_uuid: "945bd684dd674b49802e4a23cc6ac7fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "986e12bf757a465e8bdc59fae37fed20" is_pre_election: true
I20260812 01:20:41.828708 10997 raft_consensus.cc:2454] T 55e7f171b30a42eb9e4dee601edf46da P 986e12bf757a465e8bdc59fae37fed20 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 945bd684dd674b49802e4a23cc6ac7fc in term 0.
I20260812 01:20:41.828826 10996 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55e7f171b30a42eb9e4dee601edf46da" candidate_uuid: "945bd684dd674b49802e4a23cc6ac7fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "986e12bf757a465e8bdc59fae37fed20"
I20260812 01:20:41.828881 10996 raft_consensus.cc:3037] T 55e7f171b30a42eb9e4dee601edf46da P 986e12bf757a465e8bdc59fae37fed20 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:20:41.829732 10996 raft_consensus.cc:2454] T 55e7f171b30a42eb9e4dee601edf46da P 986e12bf757a465e8bdc59fae37fed20 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 945bd684dd674b49802e4a23cc6ac7fc in term 1.
W20260812 01:20:41.850963 11322 tablet.cc:2369] T 55e7f171b30a42eb9e4dee601edf46da P 6b2426cb13c543638ca6a03934d5b5eb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 01:20:41.868284 11577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:20:41.864492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36112 (local address 127.24.0.131:36545)
0812 01:20:41.864705 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0812 01:20:41.864708 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:20:41.864716 (+ 8us) server_negotiation.cc:407] Connection header received
0812 01:20:41.864764 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:20:41.864765 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:20:41.864790 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:20:41.864861 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:20:41.865090 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:20:41.865825 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:20:41.867028 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:20:41.867334 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:20:41.867407 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:20:41.867658 (+ 251us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:20:41.867801 (+ 143us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:20:41.867881 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 01:20:41.868157 (+ 276us) server_negotiation.cc:299] Negotiation successful
0812 01:20:41.868180 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":191}
I20260812 01:20:41.881542 11236 raft_consensus.cc:1261] T 55e7f171b30a42eb9e4dee601edf46da P 6b2426cb13c543638ca6a03934d5b5eb [term 1 FOLLOWER]: Refusing update from remote peer 945bd684dd674b49802e4a23cc6ac7fc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:20:41.881963 11600 consensus_queue.cc:1035] T 55e7f171b30a42eb9e4dee601edf46da P 945bd684dd674b49802e4a23cc6ac7fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b2426cb13c543638ca6a03934d5b5eb" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 34575 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:20:41.883662 10996 raft_consensus.cc:1261] T 55e7f171b30a42eb9e4dee601edf46da P 986e12bf757a465e8bdc59fae37fed20 [term 1 FOLLOWER]: Refusing update from remote peer 945bd684dd674b49802e4a23cc6ac7fc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:20:41.883919 11600 consensus_queue.cc:1035] T 55e7f171b30a42eb9e4dee601edf46da P 945bd684dd674b49802e4a23cc6ac7fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "986e12bf757a465e8bdc59fae37fed20" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 44407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:20:41.888409 11613 mvcc.cc:205] Tried to move back new op lower bound from 7317494341142843392 to 7317494340920340480. Current Snapshot: MvccSnapshot[applied={T|T < 7317494341142843392}]
I20260812 01:20:41.893114 11610 mvcc.cc:205] Tried to move back new op lower bound from 7317494341142843392 to 7317494340920340480. Current Snapshot: MvccSnapshot[applied={T|T < 7317494341142843392}]
I20260812 01:20:41.894824 10713 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57100:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:20:41.894940 10713 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57100:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:20:41.895695 10713 catalog_manager.cc:5782] T 00000000000000000000000000000000 P db2d2452aea54ae0b06a10744aa3e057: Sending DeleteTablet for 3 replicas of tablet 55e7f171b30a42eb9e4dee601edf46da
I20260812 01:20:41.896095 11188 tablet_service.cc:1514] Processing DeleteTablet for tablet 55e7f171b30a42eb9e4dee601edf46da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:20:41 UTC) from {username='kudu', principal='kudu/127.24.0.190@KRBTEST.COM'} at 127.0.0.1:35478
I20260812 01:20:41.896627 10961 tablet_service.cc:1514] Processing DeleteTablet for tablet 55e7f171b30a42eb9e4dee601edf46da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:20:41 UTC) from {username='kudu', principal='kudu/127.24.0.190@KRBTEST.COM'} at 127.0.0.1:45600
I20260812 01:20:41.896878 11412 tablet_service.cc:1514] Processing DeleteTablet for tablet 55e7f171b30a42eb9e4dee601edf46da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:20:41 UTC) from {username='kudu', principal='kudu/127.24.0.190@KRBTEST.COM'} at 127.0.0.1:36100
I20260812 01:20:41.897423 11624 tablet_replica.cc:330] T 55e7f171b30a42eb9e4dee601edf46da P 945bd684dd674b49802e4a23cc6ac7fc: stopping tablet replica
I20260812 01:20:41.897563 11624 raft_consensus.cc:2229] T 55e7f171b30a42eb9e4dee601edf46da P 945bd684dd674b49802e4a23cc6ac7fc [term 1 LEADER]: Raft consensus shutting down.
I20260812 01:20:41.897657 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 10852
I20260812 01:20:41.897727 11624 raft_consensus.cc:2258] T 55e7f171b30a42eb9e4dee601edf46da P 945bd684dd674b49802e4a23cc6ac7fc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:20:41.901183 11624 ts_tablet_manager.cc:1905] T 55e7f171b30a42eb9e4dee601edf46da P 945bd684dd674b49802e4a23cc6ac7fc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:20:41.903262 11624 ts_tablet_manager.cc:1918] T 55e7f171b30a42eb9e4dee601edf46da P 945bd684dd674b49802e4a23cc6ac7fc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 01:20:41.903333 11624 log.cc:1196] T 55e7f171b30a42eb9e4dee601edf46da P 945bd684dd674b49802e4a23cc6ac7fc: Deleting WAL directory at /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/minicluster-data/ts-2/wal/wals/55e7f171b30a42eb9e4dee601edf46da
I20260812 01:20:41.903658 11624 ts_tablet_manager.cc:1939] T 55e7f171b30a42eb9e4dee601edf46da P 945bd684dd674b49802e4a23cc6ac7fc: Deleting consensus metadata
I20260812 01:20:41.904198 10692 catalog_manager.cc:4841] TS 945bd684dd674b49802e4a23cc6ac7fc (127.24.0.131:36545): tablet 55e7f171b30a42eb9e4dee601edf46da (table test-table [id=348b06b3bca643f59e5029c626d21f2e]) successfully deleted
I20260812 01:20:41.904245 11620 tablet_replica.cc:330] T 55e7f171b30a42eb9e4dee601edf46da P 6b2426cb13c543638ca6a03934d5b5eb: stopping tablet replica
I20260812 01:20:41.904361 11620 raft_consensus.cc:2229] T 55e7f171b30a42eb9e4dee601edf46da P 6b2426cb13c543638ca6a03934d5b5eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 01:20:41.904470 11620 raft_consensus.cc:2258] T 55e7f171b30a42eb9e4dee601edf46da P 6b2426cb13c543638ca6a03934d5b5eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:20:41.905373 11620 ts_tablet_manager.cc:1905] T 55e7f171b30a42eb9e4dee601edf46da P 6b2426cb13c543638ca6a03934d5b5eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:20:41.907303 11620 ts_tablet_manager.cc:1918] T 55e7f171b30a42eb9e4dee601edf46da P 6b2426cb13c543638ca6a03934d5b5eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 01:20:41.907406 11620 log.cc:1196] T 55e7f171b30a42eb9e4dee601edf46da P 6b2426cb13c543638ca6a03934d5b5eb: Deleting WAL directory at /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786497534195564-24578-0/minicluster-data/ts-1/wal/wals/55e7f171b30a42eb9e4dee601edf46da
I20260812 01:20:41.907689 11620 ts_tablet_manager.cc:1939] T 55e7f171b30a42eb9e4dee601edf46da P 6b2426cb13c543638ca6a03934d5b5eb: Deleting consensus metadata
I20260812 01:20:41.908213 10691 catalog_manager.cc:4841] TS 6b2426cb13c543638ca6a03934d5b5eb (127.24.0.130:34575): tablet 55e7f171b30a42eb9e4dee601edf46da (table test-table [id=348b06b3bca643f59e5029c626d21f2e]) successfully deleted
W20260812 01:20:41.909480 10692 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.0.129:44407 (error 108)
I20260812 01:20:41.909754 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 11071
I20260812 01:20:41.933161 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 11333
I20260812 01:20:41.935405 11569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:20:41.932966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.129:44407 (local address 127.0.0.1:45614)
0812 01:20:41.935294 (+ 2328us) negotiation.cc:107] Waiting for socket to connect
0812 01:20:41.935359 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.129:44407: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2296}
W20260812 01:20:41.935784 10692 catalog_manager.cc:4584] TS 986e12bf757a465e8bdc59fae37fed20 (127.24.0.129:44407): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 55e7f171b30a42eb9e4dee601edf46da: Network error: Client connection negotiation failed: client connection to 127.24.0.129:44407: connect: Connection refused (error 111)
I20260812 01:20:41.941927 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 10659
I20260812 01:20:41.954018 24578 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260812 01:20:42.299938 24578 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260812 01:20:42.300122 24578 mini_ranger.cc:66] Stopping Ranger...
I20260812 01:20:42.646449 24578 mini_ranger.cc:68] Stopped Ranger
2026-08-12 01:20:42.646 UTC [9867] LOG: received smart shutdown request
2026-08-12 01:20:42.650 UTC [9867] LOG: background worker "logical replication launcher" (PID 9884) exited with exit code 1
2026-08-12 01:20:42.651 UTC [9876] LOG: shutting down
2026-08-12 01:20:42.807 UTC [9867] LOG: database system is shut down
2026-08-12T01:20:42Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (48011 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 01:20:42 dist-test-slave-t3q3 krb5kdc[11641](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 01:20:42 dist-test-slave-t3q3 krb5kdc[11641](info): set up 2 sockets
Aug 12 01:20:42 dist-test-slave-t3q3 krb5kdc[11641](info): commencing operation
krb5kdc: starting...
W20260812 01:20:44.267688 24578 mini_kdc.cc:121] Time spent starting KDC: real 1.330s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/krb5kdc/test-user.keytab.
Aug 12 01:20:44 dist-test-slave-t3q3 krb5kdc[11641](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497644, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T01:20:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:20:44Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.24.0.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.24.0.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.24.0.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/krb5kdc/rangeradmin_127.24.0.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.24.0.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/krb5kdc/rangeradmin_127.24.0.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.24.0.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.24.0.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.24.0.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/krb5kdc/rangerlookup_127.24.0.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.24.0.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/krb5kdc/rangerlookup_127.24.0.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.24.0.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.24.0.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.24.0.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/krb5kdc/HTTP_127.24.0.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.24.0.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/krb5kdc/HTTP_127.24.0.148@KRBTEST.COM.keytab.
I20260812 01:20:44.447117 24578 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskvoD5sF/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/postgres -l logfile start
2026-08-12 01:20:46.869 UTC [11687] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-12 01:20:46.869 UTC [11687] LOG: listening on IPv4 address "127.24.0.148", port 34997
2026-08-12 01:20:46.872 UTC [11687] LOG: listening on Unix socket "/tmp/.s.PGSQL.34997"
2026-08-12 01:20:46.896 UTC [11691] LOG: database system was shut down at 2026-08-12 01:20:45 UTC
2026-08-12 01:20:46.900 UTC [11687] LOG: database system is ready to accept connections
I20260812 01:20:48.132210 24578 mini_postgres.cc:96] Postgres bound to 34997
2026-08-12 01:20:48.139 UTC [11702] FATAL: database "slave" does not exist
127.24.0.148:34997 - accepting connections
I20260812 01:20:48.140542 24578 mini_ranger.cc:161] Starting Ranger...
I20260812 01:20:48.156432 24578 mini_ranger.cc:84] Created miniranger Postgres user
I20260812 01:20:48.621466 24578 mini_ranger.cc:87] Created ranger Postgres database
I20260812 01:20:48.621560 24578 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ranger-admin
2026-08-12 01:20:48,923 [I] DB FLAVOR :POSTGRES
2026-08-12 01:20:48,923 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 01:20:48,923 [I] Checking connection..
2026-08-12 01:20:48,924 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvoD5sF/build/release/bin/postgresql.jar:/tmp/dist-test-taskvoD5sF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.0.148:34997/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-12 01:20:49,163 [I] Checking connection passed.
2026-08-12 01:20:49,163 [I] --------- Verifying version history table ---------
2026-08-12 01:20:49,163 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvoD5sF/build/release/bin/postgresql.jar:/tmp/dist-test-taskvoD5sF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.0.148:34997/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 01:20:49,423 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 01:20:49,424 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvoD5sF/build/release/bin/postgresql.jar:/tmp/dist-test-taskvoD5sF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.0.148:34997/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 01:20:49,692 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 01:20:49,692 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-12 01:20:49,692 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvoD5sF/build/release/bin/postgresql.jar:/tmp/dist-test-taskvoD5sF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.0.148:34997/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskvoD5sF/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-12 01:20:49.952 UTC [11824] WARNING: there is no transaction in progress
2026-08-12 01:20:49,961 [I] create_dbversion_catalog.sql file imported successfully
2026-08-12 01:20:49,962 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvoD5sF/build/release/bin/postgresql.jar:/tmp/dist-test-taskvoD5sF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.0.148:34997/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 01:20:50,232 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-12 01:20:50,232 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 01:20:50,232 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvoD5sF/build/release/bin/postgresql.jar:/tmp/dist-test-taskvoD5sF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.0.148:34997/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-12 01:20:50,479 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvoD5sF/build/release/bin/postgresql.jar:/tmp/dist-test-taskvoD5sF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.0.148:34997/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-12 01:20:50,715 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvoD5sF/build/release/bin/postgresql.jar:/tmp/dist-test-taskvoD5sF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.0.148:34997/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-t3q3.c.gcp-upstream.internal','N') ;"
2026-08-12 01:20:50,963 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvoD5sF/build/release/bin/postgresql.jar:/tmp/dist-test-taskvoD5sF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.0.148:34997/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 01:20:51,211 [I] Table x_portal_user does not exist in database ranger
2026-08-12 01:20:51,211 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvoD5sF/build/release/bin/postgresql.jar:/tmp/dist-test-taskvoD5sF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.0.148:34997/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 01:20:51,460 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-12 01:20:51,460 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvoD5sF/build/release/bin/postgresql.jar:/tmp/dist-test-taskvoD5sF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.0.148:34997/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 01:20:51,723 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvoD5sF/build/release/bin/postgresql.jar:/tmp/dist-test-taskvoD5sF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.0.148:34997/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 01:20:51,963 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-12 01:20:51,963 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvoD5sF/build/release/bin/postgresql.jar:/tmp/dist-test-taskvoD5sF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.0.148:34997/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskvoD5sF/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-12 01:20:52.933 UTC [12012] WARNING: there is no transaction in progress
2026-08-12 01:20:52.949 UTC [12012] WARNING: there is no transaction in progress
2026-08-12 01:20:52.965 UTC [12012] WARNING: there is no transaction in progress
2026-08-12 01:20:52.980 UTC [12012] WARNING: there is no transaction in progress
2026-08-12 01:20:52.995 UTC [12012] WARNING: there is no transaction in progress
2026-08-12 01:20:53.015 UTC [12012] WARNING: there is no transaction in progress
2026-08-12 01:20:53.082 UTC [12012] WARNING: there is no transaction in progress
2026-08-12 01:20:53.089 UTC [12012] WARNING: there is no transaction in progress
2026-08-12 01:20:53.103 UTC [12012] WARNING: there is no transaction in progress
2026-08-12 01:20:53.112 UTC [12012] WARNING: there is no transaction in progress
2026-08-12 01:20:53.121 UTC [12012] WARNING: there is no transaction in progress
2026-08-12 01:20:53.134 UTC [12012] WARNING: there is no transaction in progress
2026-08-12 01:20:53.143 UTC [12012] WARNING: there is no transaction in progress
2026-08-12 01:20:53.149 UTC [12012] WARNING: there is no transaction in progress
2026-08-12 01:20:53.155 UTC [12012] WARNING: there is no transaction in progress
2026-08-12 01:20:53.160 UTC [12012] WARNING: there is no transaction in progress
2026-08-12 01:20:53,749 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-12 01:20:53,750 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvoD5sF/build/release/bin/postgresql.jar:/tmp/dist-test-taskvoD5sF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.0.148:34997/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-12 01:20:53,993 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-12 01:20:53,994 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvoD5sF/build/release/bin/postgresql.jar:/tmp/dist-test-taskvoD5sF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.0.148:34997/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 01:20:54,238 [I] Table x_portal_user already exists in database 'ranger'
2026-08-12 01:20:54,238 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvoD5sF/build/release/bin/postgresql.jar:/tmp/dist-test-taskvoD5sF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.0.148:34997/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 01:20:54,497 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-12 01:20:54,498 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvoD5sF/build/release/bin/postgresql.jar:/tmp/dist-test-taskvoD5sF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.0.148:34997/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 01:20:54,739 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvoD5sF/build/release/bin/postgresql.jar:/tmp/dist-test-taskvoD5sF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.0.148:34997/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 01:20:54,987 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvoD5sF/build/release/bin/postgresql.jar:/tmp/dist-test-taskvoD5sF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.0.148:34997/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-t3q3.c.gcp-upstream.internal';"
2026-08-12 01:20:55,244 [I] CORE_DB_SCHEMA import status has been updated
2026-08-12 01:20:55,245 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvoD5sF/build/release/bin/postgresql.jar:/tmp/dist-test-taskvoD5sF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.0.148:34997/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-12 01:20:55,521 [I] DB_PATCHES have already been applied
I20260812 01:20:55.527601 24578 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ranger-admin:/tmp/dist-test-taskvoD5sF/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskvoD5sF/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskvoD5sF/build/release/bin/postgresql.jar:/tmp/dist-test-taskvoD5sF/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260812 01:20:55.527693 24578 mini_ranger.cc:193] Using host: 127.24.0.148
I20260812 01:20:55.531668 24578 mini_ranger.cc:227] Ranger admin URL: http://127.24.0.148:51027
Aug 12, 2026 1:20:55 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 1:20:55 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 1:20:55 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ranger-admin/ews/webapp
Aug 12, 2026 1:20:55 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 12, 2026 1:20:55 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 12, 2026 1:20:56 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ranger-admin/ews/webapp].
Aug 12, 2026 1:20:56 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260812 01:20:56.913470 24578 mini_ranger.cc:160] Time spent starting Ranger: real 8.773s user 0.000s sys 0.010s
Aug 12, 2026 1:21:16 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 12, 2026 1:21:16 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 12, 2026 1:21:16 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 12, 2026 1:21:16 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 12, 2026 1:21:16 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 12, 2026 1:21:16 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 12, 2026 1:21:16 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 12, 2026 1:21:16 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 12, 2026 1:21:16 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 12, 2026 1:21:16 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 12, 2026 1:21:16 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 12, 2026 1:21:16 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 12, 2026 1:21:16 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 12, 2026 1:21:16 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 12, 2026 1:21:16 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 12, 2026 1:21:16 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 12, 2026 1:21:16 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 12, 2026 1:21:17 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260812 01:21:17.750854 24578 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.24.0.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.24.0.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.24.0.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/krb5kdc/rangerkms_127.24.0.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.24.0.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/krb5kdc/rangerkms_127.24.0.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.24.0.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.24.0.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.24.0.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/krb5kdc/HTTP_127.24.0.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.24.0.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/krb5kdc/HTTP_127.24.0.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 12 01:21:17 dist-test-slave-t3q3 krb5kdc[11641](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497677, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260812 01:21:17.841472 24578 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260812 01:21:17.851704 24578 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260812 01:21:18.663933 24578 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260812 01:21:18.664005 24578 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ranger-kms
I20260812 01:21:18.664019 24578 mini_ranger_kms.cc:227] Using postgres at 127.24.0.148:34997
2026-08-12 01:21:18,841 [I] DB FLAVOR :POSTGRES
2026-08-12 01:21:18,841 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 01:21:18,841 [I] Checking connection
2026-08-12 01:21:18,841 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvoD5sF/build/release/bin/postgresql.jar:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.0.148:34997/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-12 01:21:19,086 [I] connection success
2026-08-12 01:21:19,086 [I] --------- Verifying Ranger DB tables ---------
2026-08-12 01:21:19,086 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-12 01:21:19,087 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvoD5sF/build/release/bin/postgresql.jar:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.0.148:34997/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-12 01:21:19,330 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-12 01:21:19,331 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 01:21:19,331 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-12 01:21:19,331 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvoD5sF/build/release/bin/postgresql.jar:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.0.148:34997/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-12 01:21:19,610 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260812 01:21:20.081105 24578 mini_ranger_kms.cc:310] Created kms service
I20260812 01:21:20.270707 24578 mini_ranger_kms.cc:326] Created kudu user
I20260812 01:21:20.316231 24578 mini_ranger_kms.cc:343] Created rangerkms user
Aug 12 01:21:20 dist-test-slave-t3q3 krb5kdc[11641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497677, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.24.0.148@KRBTEST.COM
I20260812 01:21:20.717645 24578 mini_ranger_kms.cc:384] Added ranger policy
I20260812 01:21:20.717881 24578 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ranger-kms:/tmp/dist-test-taskvoD5sF/build/release/bin/postgresql.jar:/tmp/dist-test-taskvoD5sF/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskvoD5sF/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260812 01:21:20.717908 24578 mini_ranger_kms.cc:240] Using host: 127.24.0.148
I20260812 01:21:20.722604 24578 mini_ranger_kms.cc:276] Ranger KMS PID: 12325
I20260812 01:21:20.722692 24578 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.24.0.148:55653
Aug 12, 2026 1:21:21 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 1:21:21 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 1:21:21 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 12, 2026 1:21:21 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 12, 2026 1:21:22 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ranger-kms/ews/webapp].
Aug 12, 2026 1:21:22 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 12 01:21:22 dist-test-slave-t3q3 krb5kdc[11641](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786497682, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.24.0.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:21:23.496547 24578 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.655s user 0.005s sys 0.152s
I20260812 01:21:23.496656 24578 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260812 01:21:23.496708 24578 mini_ranger_kms.cc:401] 127.24.0.148:55653/kms/v1/keys
Aug 12 01:21:23 dist-test-slave-t3q3 krb5kdc[11641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497682, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.24.0.148@KRBTEST.COM for HTTP/127.24.0.148@KRBTEST.COM
Aug 12 01:21:23 dist-test-slave-t3q3 krb5kdc[11641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497682, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.24.0.148@KRBTEST.COM for HTTP/127.24.0.148@KRBTEST.COM
Aug 12 01:21:26 dist-test-slave-t3q3 krb5kdc[11641](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497686, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.24.0.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.0.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.0.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab.
Entry for principal kudu/127.24.0.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab.
I20260812 01:21:26.157227 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.190:32795
--webserver_interface=127.24.0.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab
--principal=kudu/127.24.0.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:41327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.0.190:32795
--ranger_config_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.0.148:55653/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:21:26.261983 12413 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:21:26.262202 12413 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:21:26.262224 12413 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:21:26.263381 12413 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:21:26.263422 12413 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:21:26.263437 12413 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:21:26.263522 12413 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:21:26.263540 12413 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:21:26.265187 12413 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:41327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/master-0/wal
--ranger_kms_url=127.24.0.148:55653/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.24.0.190:32795
--ranger_config_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab
--principal=kudu/127.24.0.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.190:32795
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.0.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12413
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:21:26.265451 12413 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:21:26.265704 12413 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:21:26.272131 12431 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:21:26.272115 12436 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:21:26.272251 12430 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:21:26.272253 12413 server_base.cc:1034] running on GCE node
I20260812 01:21:26.272588 12413 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:21:26.272910 12413 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:21:26.274068 12413 hybrid_clock.cc:648] HybridClock initialized: now 1786497686274052 us; error 34 us; skew 500 ppm
Aug 12 01:21:26 dist-test-slave-t3q3 krb5kdc[11641](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497686, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:21:26.276966 12413 init.cc:377] Logged in from keytab as kudu/127.24.0.190@KRBTEST.COM (short username kudu)
I20260812 01:21:26.277561 12413 webserver.cc:460] Webserver started at http://127.24.0.190:33353/ using document root <none> and password file <none>
I20260812 01:21:26.277842 12413 fs_manager.cc:359] Metadata directory not provided
I20260812 01:21:26.277904 12413 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:21:26.278048 12413 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 01:21:26 dist-test-slave-t3q3 krb5kdc[11641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497686, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.190@KRBTEST.COM for HTTP/127.24.0.148@KRBTEST.COM
I20260812 01:21:26.336217 12413 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/master-0/data/instance:
uuid: "19c171e19e4544058f8163e975c9dfa3"
format_stamp: "Formatted at 2026-08-12 01:21:26 on dist-test-slave-t3q3"
server_key: "40daeddc8f2300f59d3f7fe9d4216ae3"
server_key_iv: "fb73599315fac0171323858bef939d90"
server_key_version: "kuduclusterkey@0"
I20260812 01:21:26.336731 12413 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/master-0/wal/instance:
uuid: "19c171e19e4544058f8163e975c9dfa3"
format_stamp: "Formatted at 2026-08-12 01:21:26 on dist-test-slave-t3q3"
server_key: "40daeddc8f2300f59d3f7fe9d4216ae3"
server_key_iv: "fb73599315fac0171323858bef939d90"
server_key_version: "kuduclusterkey@0"
I20260812 01:21:26.338618 12413 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260812 01:21:26.353109 12451 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:21:26.353302 12413 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:21:26.353400 12413 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/master-0/wal
uuid: "19c171e19e4544058f8163e975c9dfa3"
format_stamp: "Formatted at 2026-08-12 01:21:26 on dist-test-slave-t3q3"
server_key: "40daeddc8f2300f59d3f7fe9d4216ae3"
server_key_iv: "fb73599315fac0171323858bef939d90"
server_key_version: "kuduclusterkey@0"
I20260812 01:21:26.353499 12413 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:21:26.378664 12413 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:21:26.380853 12413 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:21:26.381026 12413 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:21:26.386410 12413 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.190:32795
I20260812 01:21:26.386509 12559 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.190:32795 every 8 connection(s)
I20260812 01:21:26.386881 12413 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
I20260812 01:21:26.387696 12561 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:21:26.390811 12561 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 19c171e19e4544058f8163e975c9dfa3: Bootstrap starting.
I20260812 01:21:26.391547 12561 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 19c171e19e4544058f8163e975c9dfa3: Neither blocks nor log segments found. Creating new log.
I20260812 01:21:26.391870 12561 log.cc:824] T 00000000000000000000000000000000 P 19c171e19e4544058f8163e975c9dfa3: Log is configured to *not* fsync() on all Append() calls
I20260812 01:21:26.392710 12561 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 19c171e19e4544058f8163e975c9dfa3: No bootstrap required, opened a new log
I20260812 01:21:26.395469 12561 raft_consensus.cc:348] T 00000000000000000000000000000000 P 19c171e19e4544058f8163e975c9dfa3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19c171e19e4544058f8163e975c9dfa3" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 32795 } }
I20260812 01:21:26.395732 12561 raft_consensus.cc:374] T 00000000000000000000000000000000 P 19c171e19e4544058f8163e975c9dfa3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:21:26.395772 12561 raft_consensus.cc:729] T 00000000000000000000000000000000 P 19c171e19e4544058f8163e975c9dfa3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19c171e19e4544058f8163e975c9dfa3, State: Initialized, Role: FOLLOWER
I20260812 01:21:26.395771 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 12413
I20260812 01:21:26.395867 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/master-0/wal/instance
I20260812 01:21:26.396041 24578 external_mini_cluster.cc:1428] Setting key 6af0c7f6a5092adfb71555c3fe0b40c9
I20260812 01:21:26.396015 12561 consensus_queue.cc:260] T 00000000000000000000000000000000 P 19c171e19e4544058f8163e975c9dfa3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19c171e19e4544058f8163e975c9dfa3" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 32795 } }
I20260812 01:21:26.396123 12561 raft_consensus.cc:388] T 00000000000000000000000000000000 P 19c171e19e4544058f8163e975c9dfa3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:21:26.396155 12561 raft_consensus.cc:482] T 00000000000000000000000000000000 P 19c171e19e4544058f8163e975c9dfa3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:21:26.396193 12561 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 19c171e19e4544058f8163e975c9dfa3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:21:26.396876 12561 raft_consensus.cc:504] T 00000000000000000000000000000000 P 19c171e19e4544058f8163e975c9dfa3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19c171e19e4544058f8163e975c9dfa3" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 32795 } }
I20260812 01:21:26.397025 12561 leader_election.cc:302] T 00000000000000000000000000000000 P 19c171e19e4544058f8163e975c9dfa3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 19c171e19e4544058f8163e975c9dfa3; no voters:
I20260812 01:21:26.397223 12561 leader_election.cc:288] T 00000000000000000000000000000000 P 19c171e19e4544058f8163e975c9dfa3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 01:21:26.397279 12568 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 19c171e19e4544058f8163e975c9dfa3 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:21:26.397516 12568 raft_consensus.cc:686] T 00000000000000000000000000000000 P 19c171e19e4544058f8163e975c9dfa3 [term 1 LEADER]: Becoming Leader. State: Replica: 19c171e19e4544058f8163e975c9dfa3, State: Running, Role: LEADER
I20260812 01:21:26.397575 12561 sys_catalog.cc:564] T 00000000000000000000000000000000 P 19c171e19e4544058f8163e975c9dfa3 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:21:26.397665 12568 consensus_queue.cc:237] T 00000000000000000000000000000000 P 19c171e19e4544058f8163e975c9dfa3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19c171e19e4544058f8163e975c9dfa3" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 32795 } }
I20260812 01:21:26.398108 12568 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19c171e19e4544058f8163e975c9dfa3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "19c171e19e4544058f8163e975c9dfa3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19c171e19e4544058f8163e975c9dfa3" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 32795 } } }
I20260812 01:21:26.398263 12568 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19c171e19e4544058f8163e975c9dfa3 [sys.catalog]: This master's current role is: LEADER
I20260812 01:21:26.398450 12568 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19c171e19e4544058f8163e975c9dfa3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 19c171e19e4544058f8163e975c9dfa3. Latest consensus state: current_term: 1 leader_uuid: "19c171e19e4544058f8163e975c9dfa3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19c171e19e4544058f8163e975c9dfa3" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 32795 } } }
I20260812 01:21:26.398566 12568 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19c171e19e4544058f8163e975c9dfa3 [sys.catalog]: This master's current role is: LEADER
I20260812 01:21:26.398720 12583 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:21:26.399236 12583 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:21:26.399523 12561 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260812 01:21:26.401751 12583 catalog_manager.cc:1347] Generated new cluster ID: 0b0d75c21ade48428b760109124d4b0c
I20260812 01:21:26.401854 12583 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 01:21:26 dist-test-slave-t3q3 krb5kdc[11641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497686, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
I20260812 01:21:26.412287 12567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:26.396588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40046 (local address 127.24.0.190:32795)
0812 01:21:26.396784 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0812 01:21:26.396795 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 01:21:26.396831 (+ 36us) server_negotiation.cc:407] Connection header received
0812 01:21:26.396947 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:21:26.396952 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:21:26.397027 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:21:26.397167 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:21:26.398415 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:26.399810 (+ 1395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:26.400943 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:26.401518 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:26.401630 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:26.406822 (+ 5192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:21:26.406869 (+ 47us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:21:26.406869 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:21:26.406961 (+ 92us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:21:26.409511 (+ 2550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:21:26.409963 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:21:26.409967 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:21:26.409968 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:21:26.410113 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:21:26.410367 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:21:26.410367 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:21:26.410368 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:21:26.410828 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:21:26.411015 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0812 01:21:26.411562 (+ 547us) server_negotiation.cc:299] Negotiation successful
0812 01:21:26.411902 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":85,"threads_started":1}
I20260812 01:21:26.432235 12583 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:21:26.433055 12583 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:21:26.439309 12583 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 19c171e19e4544058f8163e975c9dfa3: Generated new TSK 0
I20260812 01:21:26.439611 12583 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.0.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.0.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.0.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.129.keytab.
Entry for principal kudu/127.24.0.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.129.keytab.
I20260812 01:21:27.219044 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.0.148:55653/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.0.129:0
--local_ip_for_outbound_sockets=127.24.0.129
--webserver_interface=127.24.0.129
--webserver_port=0
--tserver_master_addrs=127.24.0.190:32795
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.129.keytab
--principal=kudu/127.24.0.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:41327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:21:27.334674 12644 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:21:27.334918 12644 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:21:27.334959 12644 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:21:27.336175 12644 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:21:27.336237 12644 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:21:27.336283 12644 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.0.129
I20260812 01:21:27.337822 12644 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:41327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.24.0.148:55653/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.129.keytab
--principal=kudu/127.24.0.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.0.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.0.129
--webserver_port=0
--tserver_master_addrs=127.24.0.190:32795
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12644
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.0.129
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:21:27.338083 12644 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:21:27.338320 12644 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:21:27.343511 12654 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:21:27.343495 12655 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:21:27.343622 12661 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:21:27.343859 12644 server_base.cc:1034] running on GCE node
I20260812 01:21:27.344152 12644 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:21:27.344487 12644 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:21:27.345683 12644 hybrid_clock.cc:648] HybridClock initialized: now 1786497687345663 us; error 32 us; skew 500 ppm
Aug 12 01:21:27 dist-test-slave-t3q3 krb5kdc[11641](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497687, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:21:27.348896 12644 init.cc:377] Logged in from keytab as kudu/127.24.0.129@KRBTEST.COM (short username kudu)
I20260812 01:21:27.349421 12644 webserver.cc:460] Webserver started at http://127.24.0.129:38533/ using document root <none> and password file <none>
I20260812 01:21:27.349728 12644 fs_manager.cc:359] Metadata directory not provided
I20260812 01:21:27.349802 12644 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:21:27.349959 12644 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 01:21:27 dist-test-slave-t3q3 krb5kdc[11641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497687, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.129@KRBTEST.COM for HTTP/127.24.0.148@KRBTEST.COM
I20260812 01:21:27.368244 12644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ts-0/data/instance:
uuid: "1e21cae29c3949918ecc14a01ef9a798"
format_stamp: "Formatted at 2026-08-12 01:21:27 on dist-test-slave-t3q3"
server_key: "1d63660c5d08a2aaf31e45cf252839a7"
server_key_iv: "5c61b40adda945360fd4d5c1c67b5ae7"
server_key_version: "kuduclusterkey@0"
I20260812 01:21:27.371296 12644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ts-0/wal/instance:
uuid: "1e21cae29c3949918ecc14a01ef9a798"
format_stamp: "Formatted at 2026-08-12 01:21:27 on dist-test-slave-t3q3"
server_key: "1d63660c5d08a2aaf31e45cf252839a7"
server_key_iv: "5c61b40adda945360fd4d5c1c67b5ae7"
server_key_version: "kuduclusterkey@0"
I20260812 01:21:27.374462 12644 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.002s
I20260812 01:21:27.393422 12675 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:21:27.393613 12644 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:21:27.393695 12644 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ts-0/wal
uuid: "1e21cae29c3949918ecc14a01ef9a798"
format_stamp: "Formatted at 2026-08-12 01:21:27 on dist-test-slave-t3q3"
server_key: "1d63660c5d08a2aaf31e45cf252839a7"
server_key_iv: "5c61b40adda945360fd4d5c1c67b5ae7"
server_key_version: "kuduclusterkey@0"
I20260812 01:21:27.393790 12644 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:21:27.414229 12644 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:21:27.416525 12644 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:21:27.416706 12644 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:21:27.416977 12644 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:21:27.417414 12644 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:21:27.417470 12644 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:21:27.417542 12644 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:21:27.417575 12644 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:21:27.424649 12644 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.129:33711
I20260812 01:21:27.424744 12882 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.129:33711 every 8 connection(s)
I20260812 01:21:27.425055 12644 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ts-0/data/info.pb
I20260812 01:21:27.431042 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 12644
I20260812 01:21:27.431159 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ts-0/wal/instance
I20260812 01:21:27.431371 24578 external_mini_cluster.cc:1428] Setting key 37494c2677228880d9346fe50f02138d
Aug 12 01:21:27 dist-test-slave-t3q3 krb5kdc[11641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497687, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.129@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
I20260812 01:21:27.436954 12890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:27.425877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.129:53583 (local address 127.24.0.190:32795)
0812 01:21:27.426017 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0812 01:21:27.426019 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:21:27.426190 (+ 171us) server_negotiation.cc:407] Connection header received
0812 01:21:27.426934 (+ 744us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:21:27.426935 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:21:27.426948 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:21:27.427033 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:21:27.427879 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:27.428491 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:27.430165 (+ 1674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:27.430394 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:27.430467 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:27.433819 (+ 3352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:21:27.433820 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:21:27.433821 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:21:27.433829 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:21:27.435476 (+ 1647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:21:27.435926 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:21:27.435927 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:21:27.435928 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:21:27.435990 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:21:27.436282 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:21:27.436283 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:21:27.436283 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:21:27.436404 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:21:27.436483 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 01:21:27.436768 (+ 285us) server_negotiation.cc:299] Negotiation successful
0812 01:21:27.436855 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":75,"threads_started":1}
I20260812 01:21:27.437152 12888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:27.425765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:32795 (local address 127.24.0.129:53583)
0812 01:21:27.426010 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:27.426068 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0812 01:21:27.426783 (+ 715us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:21:27.427121 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:21:27.427126 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:21:27.427417 (+ 291us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:21:27.427741 (+ 324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:21:27.427749 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:21:27.428616 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:21:27.428617 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:21:27.430051 (+ 1434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:21:27.430054 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:21:27.431065 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:21:27.431066 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:21:27.431227 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:27.431877 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:21:27.431887 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:21:27.433708 (+ 1821us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:21:27.435583 (+ 1875us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:21:27.435591 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:21:27.435594 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:21:27.435828 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:21:27.436085 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:21:27.436086 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:21:27.436087 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:21:27.436206 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:21:27.436475 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:21:27.436482 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:21:27.436626 (+ 144us) client_negotiation.cc:769] Sending connection context
0812 01:21:27.436746 (+ 120us) client_negotiation.cc:240] Negotiation successful
0812 01:21:27.436899 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":118,"threads_started":1}
I20260812 01:21:27.437814 12884 heartbeater.cc:347] Connected to a master server at 127.24.0.190:32795
I20260812 01:21:27.437939 12884 heartbeater.cc:464] Registering TS with master...
I20260812 01:21:27.438133 12884 heartbeater.cc:511] Master 127.24.0.190:32795 requested a full tablet report, sending...
I20260812 01:21:27.438709 12468 ts_manager.cc:194] Registered new tserver with Master: 1e21cae29c3949918ecc14a01ef9a798 (127.24.0.129:33711)
I20260812 01:21:27.439324 12468 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.0.129@KRBTEST.COM'} at 127.24.0.129:53583
WARNING: no policy specified for kudu/127.24.0.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.0.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.0.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.130.keytab.
Entry for principal kudu/127.24.0.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.130.keytab.
I20260812 01:21:27.487711 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.0.148:55653/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.0.130:0
--local_ip_for_outbound_sockets=127.24.0.130
--webserver_interface=127.24.0.130
--webserver_port=0
--tserver_master_addrs=127.24.0.190:32795
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.130.keytab
--principal=kudu/127.24.0.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:41327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 01:21:27 dist-test-slave-t3q3 krb5kdc[11641](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786497687, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260812 01:21:27.591969 12897 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:21:27.592391 12897 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:21:27.592459 12897 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:21:27.594336 12897 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:21:27.594452 12897 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:21:27.594568 12897 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.0.130
I20260812 01:21:27.597048 12897 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:41327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.24.0.148:55653/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.130.keytab
--principal=kudu/127.24.0.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.0.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.0.130
--webserver_port=0
--tserver_master_addrs=127.24.0.190:32795
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12897
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.0.130
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:21:27.597335 12897 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:21:27.597728 12897 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:21:27.607863 12913 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:21:27.608245 12905 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:21:27.608626 12897 server_base.cc:1034] running on GCE node
W20260812 01:21:27.609084 12908 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:21:27.609349 12897 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:21:27.609841 12897 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:21:27.611039 12897 hybrid_clock.cc:648] HybridClock initialized: now 1786497687611015 us; error 38 us; skew 500 ppm
Aug 12 01:21:27 dist-test-slave-t3q3 krb5kdc[11641](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497687, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:21:27.614328 12897 init.cc:377] Logged in from keytab as kudu/127.24.0.130@KRBTEST.COM (short username kudu)
I20260812 01:21:27.614797 12897 webserver.cc:460] Webserver started at http://127.24.0.130:40131/ using document root <none> and password file <none>
I20260812 01:21:27.615075 12897 fs_manager.cc:359] Metadata directory not provided
I20260812 01:21:27.615149 12897 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:21:27.615319 12897 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 01:21:27 dist-test-slave-t3q3 krb5kdc[11641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497687, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.130@KRBTEST.COM for HTTP/127.24.0.148@KRBTEST.COM
I20260812 01:21:27.629601 12897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ts-1/data/instance:
uuid: "d107a76b6b9a45329560f6af8c96fe62"
format_stamp: "Formatted at 2026-08-12 01:21:27 on dist-test-slave-t3q3"
server_key: "1e0504f3829e38c7034aab3d020569b3"
server_key_iv: "86adfbe4fae9629b893c893a0d9dee68"
server_key_version: "kuduclusterkey@0"
I20260812 01:21:27.630148 12897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ts-1/wal/instance:
uuid: "d107a76b6b9a45329560f6af8c96fe62"
format_stamp: "Formatted at 2026-08-12 01:21:27 on dist-test-slave-t3q3"
server_key: "1e0504f3829e38c7034aab3d020569b3"
server_key_iv: "86adfbe4fae9629b893c893a0d9dee68"
server_key_version: "kuduclusterkey@0"
I20260812 01:21:27.632300 12897 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260812 01:21:27.649022 12931 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:21:27.652400 12897 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.001s sys 0.000s
I20260812 01:21:27.652549 12897 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ts-1/wal
uuid: "d107a76b6b9a45329560f6af8c96fe62"
format_stamp: "Formatted at 2026-08-12 01:21:27 on dist-test-slave-t3q3"
server_key: "1e0504f3829e38c7034aab3d020569b3"
server_key_iv: "86adfbe4fae9629b893c893a0d9dee68"
server_key_version: "kuduclusterkey@0"
I20260812 01:21:27.652670 12897 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:21:27.664927 12897 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:21:27.667492 12897 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:21:27.667712 12897 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:21:27.668098 12897 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:21:27.668493 12897 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:21:27.668581 12897 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:21:27.668646 12897 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:21:27.668692 12897 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:21:27.675163 12897 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.130:39033
I20260812 01:21:27.675665 12897 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ts-1/data/info.pb
I20260812 01:21:27.678198 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 12897
I20260812 01:21:27.678289 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ts-1/wal/instance
I20260812 01:21:27.678500 24578 external_mini_cluster.cc:1428] Setting key 342f2ed9a8b412ed29608117282f4399
I20260812 01:21:27.686220 13133 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.130:39033 every 8 connection(s)
Aug 12 01:21:27 dist-test-slave-t3q3 krb5kdc[11641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497687, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.130@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.24.0.131@KRBTEST.COM; defaulting to no policy
I20260812 01:21:27.703881 12890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:27.687313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.130:58509 (local address 127.24.0.190:32795)
0812 01:21:27.687367 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0812 01:21:27.687369 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:21:27.692403 (+ 5034us) server_negotiation.cc:407] Connection header received
0812 01:21:27.693202 (+ 799us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:21:27.693204 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:21:27.693223 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:21:27.693256 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:21:27.694198 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:27.695012 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:27.696595 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:27.696927 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:27.696976 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:27.700208 (+ 3232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:21:27.700209 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:21:27.700210 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:21:27.700218 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:21:27.702001 (+ 1783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:21:27.702609 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:21:27.702611 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:21:27.702611 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:21:27.702678 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:21:27.703051 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:21:27.703052 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:21:27.703052 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:21:27.703199 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:21:27.703270 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 01:21:27.703681 (+ 411us) server_negotiation.cc:299] Negotiation successful
0812 01:21:27.703781 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 01:21:27.704123 13146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:27.687321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:32795 (local address 127.24.0.130:58509)
0812 01:21:27.692219 (+ 4898us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:27.692284 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0812 01:21:27.693054 (+ 770us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:21:27.693465 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:21:27.693470 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:21:27.693773 (+ 303us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:21:27.694076 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:21:27.694082 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:21:27.695139 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:21:27.695140 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:21:27.696472 (+ 1332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:21:27.696475 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:21:27.697084 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:21:27.697085 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:21:27.697226 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:27.697956 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:21:27.697969 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:21:27.700082 (+ 2113us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:21:27.702110 (+ 2028us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:21:27.702119 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:21:27.702122 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:21:27.702485 (+ 363us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:21:27.702748 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:21:27.702750 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:21:27.702750 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:21:27.702917 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:21:27.703287 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:21:27.703296 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:21:27.703488 (+ 192us) client_negotiation.cc:769] Sending connection context
0812 01:21:27.703617 (+ 129us) client_negotiation.cc:240] Negotiation successful
0812 01:21:27.703801 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4837,"thread_start_us":105,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.0.131@KRBTEST.COM" created.
I20260812 01:21:27.704900 13136 heartbeater.cc:347] Connected to a master server at 127.24.0.190:32795
I20260812 01:21:27.705003 13136 heartbeater.cc:464] Registering TS with master...
I20260812 01:21:27.705205 13136 heartbeater.cc:511] Master 127.24.0.190:32795 requested a full tablet report, sending...
I20260812 01:21:27.705612 12468 ts_manager.cc:194] Registered new tserver with Master: d107a76b6b9a45329560f6af8c96fe62 (127.24.0.130:39033)
I20260812 01:21:27.706092 12468 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.0.130@KRBTEST.COM'} at 127.24.0.130:58509
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.0.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.131.keytab.
Entry for principal kudu/127.24.0.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.131.keytab.
I20260812 01:21:27.723304 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.0.148:55653/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.0.131:0
--local_ip_for_outbound_sockets=127.24.0.131
--webserver_interface=127.24.0.131
--webserver_port=0
--tserver_master_addrs=127.24.0.190:32795
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.131.keytab
--principal=kudu/127.24.0.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:41327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:21:27.865109 13149 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:21:27.865485 13149 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:21:27.865547 13149 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:21:27.866744 13149 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:21:27.866820 13149 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:21:27.866868 13149 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.0.131
I20260812 01:21:27.868425 13149 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:41327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.24.0.148:55653/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.131.keytab
--principal=kudu/127.24.0.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.0.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.0.131
--webserver_port=0
--tserver_master_addrs=127.24.0.190:32795
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13149
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.0.131
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:21:27.868722 13149 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:21:27.868973 13149 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:21:27.874636 13161 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:21:27.874792 13164 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:21:27.874682 13149 server_base.cc:1034] running on GCE node
W20260812 01:21:27.874683 13159 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:21:27.875110 13149 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:21:27.875437 13149 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:21:27.876610 13149 hybrid_clock.cc:648] HybridClock initialized: now 1786497687876584 us; error 43 us; skew 500 ppm
Aug 12 01:21:27 dist-test-slave-t3q3 krb5kdc[11641](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497687, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:21:27.879225 13149 init.cc:377] Logged in from keytab as kudu/127.24.0.131@KRBTEST.COM (short username kudu)
I20260812 01:21:27.879747 13149 webserver.cc:460] Webserver started at http://127.24.0.131:38643/ using document root <none> and password file <none>
I20260812 01:21:27.880000 13149 fs_manager.cc:359] Metadata directory not provided
I20260812 01:21:27.880092 13149 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:21:27.880255 13149 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 01:21:27 dist-test-slave-t3q3 krb5kdc[11641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497687, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.131@KRBTEST.COM for HTTP/127.24.0.148@KRBTEST.COM
I20260812 01:21:27.897300 13149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ts-2/data/instance:
uuid: "0d72c8d421964ec1ad457c7f7b815931"
format_stamp: "Formatted at 2026-08-12 01:21:27 on dist-test-slave-t3q3"
server_key: "93aaff582697070a256df38333c4af0d"
server_key_iv: "3fc878f276709c1fa4c3f29e264b770b"
server_key_version: "kuduclusterkey@0"
I20260812 01:21:27.898036 13149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ts-2/wal/instance:
uuid: "0d72c8d421964ec1ad457c7f7b815931"
format_stamp: "Formatted at 2026-08-12 01:21:27 on dist-test-slave-t3q3"
server_key: "93aaff582697070a256df38333c4af0d"
server_key_iv: "3fc878f276709c1fa4c3f29e264b770b"
server_key_version: "kuduclusterkey@0"
I20260812 01:21:27.912569 13149 fs_manager.cc:691] Time spent creating directory manager: real 0.014s user 0.000s sys 0.002s
I20260812 01:21:27.936226 13176 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:21:27.936432 13149 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.001s sys 0.000s
I20260812 01:21:27.936538 13149 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ts-2/wal
uuid: "0d72c8d421964ec1ad457c7f7b815931"
format_stamp: "Formatted at 2026-08-12 01:21:27 on dist-test-slave-t3q3"
server_key: "93aaff582697070a256df38333c4af0d"
server_key_iv: "3fc878f276709c1fa4c3f29e264b770b"
server_key_version: "kuduclusterkey@0"
I20260812 01:21:27.936650 13149 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:21:27.954442 13149 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:21:27.957214 13149 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:21:27.957520 13149 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:21:27.957803 13149 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:21:27.958263 13149 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:21:27.958352 13149 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:21:27.958437 13149 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:21:27.958501 13149 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:21:27.968160 13149 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.131:40871
I20260812 01:21:27.968554 13149 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ts-2/data/info.pb
I20260812 01:21:27.969270 13352 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.131:40871 every 8 connection(s)
I20260812 01:21:27.974308 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 13149
I20260812 01:21:27.974433 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ts-2/wal/instance
I20260812 01:21:27.974632 24578 external_mini_cluster.cc:1428] Setting key b980d5720cbd2d200f47d9a919ee8527
Aug 12 01:21:27 dist-test-slave-t3q3 krb5kdc[11641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497687, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.131@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
I20260812 01:21:27.978510 12890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:27.969374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.131:57819 (local address 127.24.0.190:32795)
0812 01:21:27.969427 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0812 01:21:27.969429 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:21:27.969762 (+ 333us) server_negotiation.cc:407] Connection header received
0812 01:21:27.970303 (+ 541us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:21:27.970305 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:21:27.970319 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:21:27.970370 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:21:27.970945 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:27.971460 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:27.972587 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:27.972867 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:27.972950 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:27.975466 (+ 2516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:21:27.975468 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:21:27.975468 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:21:27.975475 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:21:27.977066 (+ 1591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:21:27.977507 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:21:27.977508 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:21:27.977509 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:21:27.977573 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:21:27.977849 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:21:27.977850 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:21:27.977850 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:21:27.977969 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:21:27.978048 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 01:21:27.978334 (+ 286us) server_negotiation.cc:299] Negotiation successful
0812 01:21:27.978422 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 01:21:27.978716 13363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:27.969380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:32795 (local address 127.24.0.131:57819)
0812 01:21:27.969626 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:27.969665 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 01:21:27.970186 (+ 521us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:21:27.970442 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:21:27.970446 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:21:27.970654 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:21:27.970847 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:21:27.970851 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:21:27.971569 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:21:27.971571 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:21:27.972486 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:21:27.972489 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:21:27.972970 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:21:27.972971 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:21:27.973120 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:27.973689 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:21:27.973699 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:21:27.975353 (+ 1654us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:21:27.977184 (+ 1831us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:21:27.977191 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:21:27.977193 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:21:27.977426 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:21:27.977644 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:21:27.977645 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:21:27.977645 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:21:27.977772 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:21:27.978039 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:21:27.978046 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:21:27.978191 (+ 145us) client_negotiation.cc:769] Sending connection context
0812 01:21:27.978300 (+ 109us) client_negotiation.cc:240] Negotiation successful
0812 01:21:27.978448 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":87,"threads_started":1}
I20260812 01:21:27.979180 13354 heartbeater.cc:347] Connected to a master server at 127.24.0.190:32795
I20260812 01:21:27.979279 13354 heartbeater.cc:464] Registering TS with master...
I20260812 01:21:27.979468 13354 heartbeater.cc:511] Master 127.24.0.190:32795 requested a full tablet report, sending...
I20260812 01:21:27.979875 12468 ts_manager.cc:194] Registered new tserver with Master: 0d72c8d421964ec1ad457c7f7b815931 (127.24.0.131:40871)
I20260812 01:21:27.980329 12468 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.0.131@KRBTEST.COM'} at 127.24.0.131:57819
I20260812 01:21:27.986248 24578 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 01:21:27.986821 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 12644
I20260812 01:21:28.024959 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 12897
I20260812 01:21:28.039412 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 13149
I20260812 01:21:28.087574 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 12413
I20260812 01:21:28.104328 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.190:32795
--webserver_interface=127.24.0.190
--webserver_port=33353
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab
--principal=kudu/127.24.0.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:41327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.0.190:32795
--ranger_config_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.0.148:55653/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.24.0.148:55653/kms with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:21:28.187856 13365 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:21:28.188131 13365 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:21:28.188172 13365 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:21:28.189271 13365 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:21:28.189308 13365 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:21:28.189323 13365 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:21:28.189335 13365 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:21:28.189348 13365 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:21:28.190886 13365 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:41327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.24.0.148:55653/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.24.0.190:32795
--ranger_config_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab
--principal=kudu/127.24.0.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.190:32795
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.0.190
--webserver_port=33353
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13365
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:21:28.191087 13365 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:21:28.191262 13365 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:21:28.196317 13376 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:21:28.196429 13379 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:21:28.196553 13365 server_base.cc:1034] running on GCE node
W20260812 01:21:28.196455 13374 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:21:28.196954 13365 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:21:28.197335 13365 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:21:28.198535 13365 hybrid_clock.cc:648] HybridClock initialized: now 1786497688198524 us; error 56 us; skew 500 ppm
Aug 12 01:21:28 dist-test-slave-t3q3 krb5kdc[11641](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497688, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:21:28.201915 13365 init.cc:377] Logged in from keytab as kudu/127.24.0.190@KRBTEST.COM (short username kudu)
I20260812 01:21:28.202443 13365 webserver.cc:460] Webserver started at http://127.24.0.190:33353/ using document root <none> and password file <none>
I20260812 01:21:28.202689 13365 fs_manager.cc:359] Metadata directory not provided
I20260812 01:21:28.202769 13365 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 01:21:28 dist-test-slave-t3q3 krb5kdc[11641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497688, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.190@KRBTEST.COM for HTTP/127.24.0.148@KRBTEST.COM
I20260812 01:21:28.279810 13365 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:21:28.280720 13391 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:21:28.280905 13365 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 01:21:28.281023 13365 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/master-0/wal
uuid: "19c171e19e4544058f8163e975c9dfa3"
format_stamp: "Formatted at 2026-08-12 01:21:26 on dist-test-slave-t3q3"
server_key: "40daeddc8f2300f59d3f7fe9d4216ae3"
server_key_iv: "fb73599315fac0171323858bef939d90"
server_key_version: "kuduclusterkey@0"
I20260812 01:21:28.281414 13365 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:21:28.310870 13365 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:21:28.312731 13365 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:21:28.312911 13365 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:21:28.317406 13365 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.190:32795
I20260812 01:21:28.317461 13489 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.190:32795 every 8 connection(s)
I20260812 01:21:28.317840 13365 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
I20260812 01:21:28.319046 13491 sys_catalog.cc:263] Verifying existing consensus state
I20260812 01:21:28.319717 13491 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 19c171e19e4544058f8163e975c9dfa3: Bootstrap starting.
I20260812 01:21:28.321560 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 13365
I20260812 01:21:28.321921 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.0.148:55653/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.0.129:33711
--local_ip_for_outbound_sockets=127.24.0.129
--tserver_master_addrs=127.24.0.190:32795
--webserver_port=38533
--webserver_interface=127.24.0.129
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.129.keytab
--principal=kudu/127.24.0.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:41327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.24.0.148:55653/kms with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 01:21:28.328621 13491 log.cc:824] T 00000000000000000000000000000000 P 19c171e19e4544058f8163e975c9dfa3: Log is configured to *not* fsync() on all Append() calls
I20260812 01:21:28.330406 13491 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 19c171e19e4544058f8163e975c9dfa3: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 01:21:28.330726 13491 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 19c171e19e4544058f8163e975c9dfa3: Bootstrap complete.
I20260812 01:21:28.333318 13491 raft_consensus.cc:348] T 00000000000000000000000000000000 P 19c171e19e4544058f8163e975c9dfa3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19c171e19e4544058f8163e975c9dfa3" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 32795 } }
I20260812 01:21:28.333628 13491 raft_consensus.cc:729] T 00000000000000000000000000000000 P 19c171e19e4544058f8163e975c9dfa3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19c171e19e4544058f8163e975c9dfa3, State: Initialized, Role: FOLLOWER
I20260812 01:21:28.333799 13491 consensus_queue.cc:260] T 00000000000000000000000000000000 P 19c171e19e4544058f8163e975c9dfa3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19c171e19e4544058f8163e975c9dfa3" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 32795 } }
I20260812 01:21:28.333871 13491 raft_consensus.cc:388] T 00000000000000000000000000000000 P 19c171e19e4544058f8163e975c9dfa3 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:21:28.333921 13491 raft_consensus.cc:482] T 00000000000000000000000000000000 P 19c171e19e4544058f8163e975c9dfa3 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:21:28.333971 13491 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 19c171e19e4544058f8163e975c9dfa3 [term 1 FOLLOWER]: Advancing to term 2
I20260812 01:21:28.334964 13491 raft_consensus.cc:504] T 00000000000000000000000000000000 P 19c171e19e4544058f8163e975c9dfa3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19c171e19e4544058f8163e975c9dfa3" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 32795 } }
I20260812 01:21:28.335055 13491 leader_election.cc:302] T 00000000000000000000000000000000 P 19c171e19e4544058f8163e975c9dfa3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 19c171e19e4544058f8163e975c9dfa3; no voters:
I20260812 01:21:28.335273 13491 leader_election.cc:288] T 00000000000000000000000000000000 P 19c171e19e4544058f8163e975c9dfa3 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 01:21:28.335338 13496 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 19c171e19e4544058f8163e975c9dfa3 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 01:21:28.335516 13496 raft_consensus.cc:686] T 00000000000000000000000000000000 P 19c171e19e4544058f8163e975c9dfa3 [term 2 LEADER]: Becoming Leader. State: Replica: 19c171e19e4544058f8163e975c9dfa3, State: Running, Role: LEADER
I20260812 01:21:28.335582 13491 sys_catalog.cc:564] T 00000000000000000000000000000000 P 19c171e19e4544058f8163e975c9dfa3 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:21:28.335644 13496 consensus_queue.cc:237] T 00000000000000000000000000000000 P 19c171e19e4544058f8163e975c9dfa3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19c171e19e4544058f8163e975c9dfa3" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 32795 } }
I20260812 01:21:28.336002 13497 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19c171e19e4544058f8163e975c9dfa3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "19c171e19e4544058f8163e975c9dfa3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19c171e19e4544058f8163e975c9dfa3" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 32795 } } }
I20260812 01:21:28.336128 13497 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19c171e19e4544058f8163e975c9dfa3 [sys.catalog]: This master's current role is: LEADER
I20260812 01:21:28.336015 13499 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19c171e19e4544058f8163e975c9dfa3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 19c171e19e4544058f8163e975c9dfa3. Latest consensus state: current_term: 2 leader_uuid: "19c171e19e4544058f8163e975c9dfa3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19c171e19e4544058f8163e975c9dfa3" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 32795 } } }
I20260812 01:21:28.336200 13499 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19c171e19e4544058f8163e975c9dfa3 [sys.catalog]: This master's current role is: LEADER
I20260812 01:21:28.336434 13509 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:21:28.336932 13509 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:21:28.337073 13509 catalog_manager.cc:1259] Loaded cluster ID: 0b0d75c21ade48428b760109124d4b0c
I20260812 01:21:28.337109 13509 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 01:21:28.337343 13491 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260812 01:21:28.340881 13509 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:21:28.341545 13509 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 19c171e19e4544058f8163e975c9dfa3: Loaded TSK: 0
I20260812 01:21:28.341843 13509 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 01:21:28.422947 13493 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:21:28.423174 13493 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:21:28.423218 13493 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:21:28.424456 13493 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:21:28.424523 13493 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:21:28.424567 13493 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.0.129
I20260812 01:21:28.426105 13493 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:41327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.24.0.148:55653/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.129.keytab
--principal=kudu/127.24.0.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.0.129:33711
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.0.129
--webserver_port=38533
--tserver_master_addrs=127.24.0.190:32795
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13493
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.0.129
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:21:28.426374 13493 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:21:28.426682 13493 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:21:28.432345 13567 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:21:28.432467 13560 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:21:28.432356 13563 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:21:28.433332 13493 server_base.cc:1034] running on GCE node
I20260812 01:21:28.433610 13493 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:21:28.433893 13493 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:21:28.441267 13493 hybrid_clock.cc:648] HybridClock initialized: now 1786497688441251 us; error 33 us; skew 500 ppm
Aug 12 01:21:28 dist-test-slave-t3q3 krb5kdc[11641](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497688, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:21:28.443928 13493 init.cc:377] Logged in from keytab as kudu/127.24.0.129@KRBTEST.COM (short username kudu)
I20260812 01:21:28.444502 13493 webserver.cc:460] Webserver started at http://127.24.0.129:38533/ using document root <none> and password file <none>
I20260812 01:21:28.444775 13493 fs_manager.cc:359] Metadata directory not provided
I20260812 01:21:28.444876 13493 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 01:21:28 dist-test-slave-t3q3 krb5kdc[11641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497688, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.129@KRBTEST.COM for HTTP/127.24.0.148@KRBTEST.COM
I20260812 01:21:28.462559 13493 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260812 01:21:28.464783 13576 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:21:28.465013 13493 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.000s
I20260812 01:21:28.465098 13493 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ts-0/wal
uuid: "1e21cae29c3949918ecc14a01ef9a798"
format_stamp: "Formatted at 2026-08-12 01:21:27 on dist-test-slave-t3q3"
server_key: "1d63660c5d08a2aaf31e45cf252839a7"
server_key_iv: "5c61b40adda945360fd4d5c1c67b5ae7"
server_key_version: "kuduclusterkey@0"
I20260812 01:21:28.465564 13493 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:21:28.478736 13493 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:21:28.481532 13493 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:21:28.481765 13493 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:21:28.482091 13493 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:21:28.482539 13493 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:21:28.482635 13493 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:21:28.482738 13493 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:21:28.482812 13493 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:21:28.490434 13493 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.129:33711
I20260812 01:21:28.491016 13493 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ts-0/data/info.pb
I20260812 01:21:28.494725 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 13493
I20260812 01:21:28.495026 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.0.148:55653/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.0.130:39033
--local_ip_for_outbound_sockets=127.24.0.130
--tserver_master_addrs=127.24.0.190:32795
--webserver_port=40131
--webserver_interface=127.24.0.130
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.130.keytab
--principal=kudu/127.24.0.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:41327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.24.0.148:55653/kms with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 01:21:28.492791 13777 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.129:33711 every 8 connection(s)
Aug 12 01:21:28 dist-test-slave-t3q3 krb5kdc[11641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497688, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.129@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
I20260812 01:21:28.511607 13781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:28.493931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:32795 (local address 127.24.0.129:56899)
0812 01:21:28.497660 (+ 3729us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:28.497730 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0812 01:21:28.498304 (+ 574us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:21:28.498892 (+ 588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:21:28.498896 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:21:28.499184 (+ 288us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:21:28.499489 (+ 305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:21:28.499496 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:21:28.501257 (+ 1761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:21:28.501258 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:21:28.502698 (+ 1440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:21:28.502702 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:21:28.503781 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:21:28.503782 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:21:28.503943 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:28.504673 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:21:28.504686 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:21:28.506565 (+ 1879us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:21:28.509326 (+ 2761us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:21:28.509333 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:21:28.509336 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:21:28.509694 (+ 358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:21:28.510025 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:21:28.510026 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:21:28.510027 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:21:28.510181 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:21:28.510777 (+ 596us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:21:28.510784 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:21:28.510964 (+ 180us) client_negotiation.cc:769] Sending connection context
0812 01:21:28.511089 (+ 125us) client_negotiation.cc:240] Negotiation successful
0812 01:21:28.511298 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3672,"thread_start_us":90,"threads_started":1}
I20260812 01:21:28.512373 13782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:28.494353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.129:56899 (local address 127.24.0.190:32795)
0812 01:21:28.498488 (+ 4135us) server_negotiation.cc:206] Beginning negotiation
0812 01:21:28.498496 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 01:21:28.498524 (+ 28us) server_negotiation.cc:407] Connection header received
0812 01:21:28.498646 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:21:28.498650 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:21:28.498691 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:21:28.498850 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:21:28.499643 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:28.501115 (+ 1472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:28.502809 (+ 1694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:28.503533 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:28.503652 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:28.506734 (+ 3082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:21:28.506739 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:21:28.506740 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:21:28.506755 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:21:28.509202 (+ 2447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:21:28.509806 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:21:28.509809 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:21:28.509810 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:21:28.509937 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:21:28.510282 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:21:28.510283 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:21:28.510284 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:21:28.510641 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:21:28.510741 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0812 01:21:28.511947 (+ 1206us) server_negotiation.cc:299] Negotiation successful
0812 01:21:28.512097 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4083,"thread_start_us":87,"threads_started":1}
I20260812 01:21:28.513185 13779 heartbeater.cc:347] Connected to a master server at 127.24.0.190:32795
I20260812 01:21:28.513305 13779 heartbeater.cc:464] Registering TS with master...
I20260812 01:21:28.513526 13779 heartbeater.cc:511] Master 127.24.0.190:32795 requested a full tablet report, sending...
W20260812 01:21:28.514447 13779 heartbeater.cc:644] Failed to heartbeat to 127.24.0.190:32795 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 01:21:28.514515 13779 heartbeater.cc:464] Registering TS with master...
I20260812 01:21:28.514575 13779 heartbeater.cc:511] Master 127.24.0.190:32795 requested a full tablet report, sending...
I20260812 01:21:28.514966 13779 heartbeater.cc:464] Registering TS with master...
I20260812 01:21:28.515090 13779 heartbeater.cc:511] Master 127.24.0.190:32795 requested a full tablet report, sending...
W20260812 01:21:28.515525 13779 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 01:21:28.601326 13784 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:21:28.601610 13784 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:21:28.601653 13784 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:21:28.603135 13784 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:21:28.603200 13784 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:21:28.603246 13784 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.0.130
I20260812 01:21:28.605119 13784 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:41327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.24.0.148:55653/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.130.keytab
--principal=kudu/127.24.0.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.0.130:39033
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.0.130
--webserver_port=40131
--tserver_master_addrs=127.24.0.190:32795
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13784
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.0.130
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:21:28.605372 13784 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:21:28.605719 13784 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:21:28.612262 13793 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:21:28.612318 13794 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:21:28.612731 13797 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:21:28.612926 13784 server_base.cc:1034] running on GCE node
I20260812 01:21:28.613157 13784 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:21:28.613446 13784 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:21:28.614634 13784 hybrid_clock.cc:648] HybridClock initialized: now 1786497688614604 us; error 44 us; skew 500 ppm
Aug 12 01:21:28 dist-test-slave-t3q3 krb5kdc[11641](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497688, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:21:28.617985 13784 init.cc:377] Logged in from keytab as kudu/127.24.0.130@KRBTEST.COM (short username kudu)
I20260812 01:21:28.618539 13784 webserver.cc:460] Webserver started at http://127.24.0.130:40131/ using document root <none> and password file <none>
I20260812 01:21:28.618866 13784 fs_manager.cc:359] Metadata directory not provided
I20260812 01:21:28.619006 13784 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 01:21:28 dist-test-slave-t3q3 krb5kdc[11641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497688, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.130@KRBTEST.COM for HTTP/127.24.0.148@KRBTEST.COM
I20260812 01:21:28.638285 13784 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:21:28.640995 13810 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:21:28.641180 13784 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.000s
I20260812 01:21:28.641317 13784 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ts-1/wal
uuid: "d107a76b6b9a45329560f6af8c96fe62"
format_stamp: "Formatted at 2026-08-12 01:21:27 on dist-test-slave-t3q3"
server_key: "1e0504f3829e38c7034aab3d020569b3"
server_key_iv: "86adfbe4fae9629b893c893a0d9dee68"
server_key_version: "kuduclusterkey@0"
I20260812 01:21:28.641827 13784 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:21:28.658695 13784 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:21:28.661420 13784 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:21:28.661652 13784 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:21:28.661970 13784 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:21:28.662408 13784 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:21:28.662499 13784 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:21:28.662567 13784 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:21:28.662619 13784 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:21:28.669269 13784 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.130:39033
I20260812 01:21:28.669911 13784 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ts-1/data/info.pb
I20260812 01:21:28.675294 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 13784
I20260812 01:21:28.675686 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.0.148:55653/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.0.131:40871
--local_ip_for_outbound_sockets=127.24.0.131
--tserver_master_addrs=127.24.0.190:32795
--webserver_port=38643
--webserver_interface=127.24.0.131
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.131.keytab
--principal=kudu/127.24.0.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:41327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.24.0.148:55653/kms with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 01:21:28.676182 14025 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.130:39033 every 8 connection(s)
Aug 12 01:21:28 dist-test-slave-t3q3 krb5kdc[11641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497688, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.130@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
I20260812 01:21:28.690910 13782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:28.677686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.130:51929 (local address 127.24.0.190:32795)
0812 01:21:28.677742 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 01:21:28.677745 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:21:28.677984 (+ 239us) server_negotiation.cc:407] Connection header received
0812 01:21:28.678789 (+ 805us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:21:28.678791 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:21:28.678807 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:21:28.678841 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:21:28.679718 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:28.680562 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:28.682229 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:28.682605 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:28.682657 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:28.685575 (+ 2918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:21:28.685577 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:21:28.685577 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:21:28.685586 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:21:28.687588 (+ 2002us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:21:28.689749 (+ 2161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:21:28.689751 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:21:28.689752 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:21:28.689823 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:21:28.690127 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:21:28.690129 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:21:28.690129 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:21:28.690273 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:21:28.690317 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0812 01:21:28.690705 (+ 388us) server_negotiation.cc:299] Negotiation successful
0812 01:21:28.690806 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 01:21:28.691747 14029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:28.676422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:32795 (local address 127.24.0.130:51929)
0812 01:21:28.677827 (+ 1405us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:28.677885 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0812 01:21:28.678641 (+ 756us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:21:28.679037 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:21:28.679043 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:21:28.679299 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:21:28.679598 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:21:28.679604 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:21:28.680696 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:21:28.680697 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:21:28.682111 (+ 1414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:21:28.682114 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:21:28.682786 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:21:28.682787 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:21:28.682948 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:28.683530 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:21:28.683551 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:21:28.685447 (+ 1896us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:21:28.687741 (+ 2294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:21:28.687749 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:21:28.687752 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:21:28.688129 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:21:28.689902 (+ 1773us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:21:28.689902 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:21:28.689903 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:21:28.690047 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:21:28.690364 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:21:28.690370 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:21:28.690550 (+ 180us) client_negotiation.cc:769] Sending connection context
0812 01:21:28.691301 (+ 751us) client_negotiation.cc:240] Negotiation successful
0812 01:21:28.691479 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1355,"thread_start_us":101,"threads_started":1}
I20260812 01:21:28.692476 14026 heartbeater.cc:347] Connected to a master server at 127.24.0.190:32795
I20260812 01:21:28.692641 14026 heartbeater.cc:464] Registering TS with master...
I20260812 01:21:28.692883 14026 heartbeater.cc:511] Master 127.24.0.190:32795 requested a full tablet report, sending...
W20260812 01:21:28.693545 14026 heartbeater.cc:644] Failed to heartbeat to 127.24.0.190:32795 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 01:21:28.693665 14026 heartbeater.cc:464] Registering TS with master...
I20260812 01:21:28.693764 14026 heartbeater.cc:511] Master 127.24.0.190:32795 requested a full tablet report, sending...
I20260812 01:21:28.694068 14026 heartbeater.cc:464] Registering TS with master...
I20260812 01:21:28.694175 14026 heartbeater.cc:511] Master 127.24.0.190:32795 requested a full tablet report, sending...
W20260812 01:21:28.694466 14026 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 01:21:28.786535 14031 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:21:28.786829 14031 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:21:28.786870 14031 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:21:28.788651 14031 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:21:28.788744 14031 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:21:28.788802 14031 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.0.131
I20260812 01:21:28.790970 14031 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:41327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.24.0.148:55653/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.131.keytab
--principal=kudu/127.24.0.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.0.131:40871
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.0.131
--webserver_port=38643
--tserver_master_addrs=127.24.0.190:32795
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14031
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.0.131
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:21:28.791371 14031 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:21:28.791667 14031 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:21:28.798508 14038 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:21:28.798525 14043 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:21:28.798950 14045 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:21:28.799249 14031 server_base.cc:1034] running on GCE node
I20260812 01:21:28.799612 14031 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:21:28.799909 14031 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:21:28.803196 14031 hybrid_clock.cc:648] HybridClock initialized: now 1786497688803163 us; error 59 us; skew 500 ppm
Aug 12 01:21:28 dist-test-slave-t3q3 krb5kdc[11641](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497688, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:21:28.806650 14031 init.cc:377] Logged in from keytab as kudu/127.24.0.131@KRBTEST.COM (short username kudu)
I20260812 01:21:28.807269 14031 webserver.cc:460] Webserver started at http://127.24.0.131:38643/ using document root <none> and password file <none>
I20260812 01:21:28.807580 14031 fs_manager.cc:359] Metadata directory not provided
I20260812 01:21:28.807721 14031 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 01:21:28 dist-test-slave-t3q3 krb5kdc[11641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497688, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.131@KRBTEST.COM for HTTP/127.24.0.148@KRBTEST.COM
I20260812 01:21:28.822760 14031 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:21:28.823884 14057 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:21:28.824137 14031 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 01:21:28.824227 14031 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ts-2/wal
uuid: "0d72c8d421964ec1ad457c7f7b815931"
format_stamp: "Formatted at 2026-08-12 01:21:27 on dist-test-slave-t3q3"
server_key: "93aaff582697070a256df38333c4af0d"
server_key_iv: "3fc878f276709c1fa4c3f29e264b770b"
server_key_version: "kuduclusterkey@0"
I20260812 01:21:28.824685 14031 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:21:28.840358 14031 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:21:28.842840 14031 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:21:28.843007 14031 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:21:28.843226 14031 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:21:28.843673 14031 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:21:28.843737 14031 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:21:28.843771 14031 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:21:28.843832 14031 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:21:28.851177 14031 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.131:40871
I20260812 01:21:28.851683 14031 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ts-2/data/info.pb
I20260812 01:21:28.853433 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 14031
Aug 12 01:21:28 dist-test-slave-t3q3 krb5kdc[11641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497688, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.131@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
I20260812 01:21:28.852692 14263 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.131:40871 every 8 connection(s)
I20260812 01:21:28.864619 13782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:28.853788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.131:54383 (local address 127.24.0.190:32795)
0812 01:21:28.853934 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0812 01:21:28.853936 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:21:28.853949 (+ 13us) server_negotiation.cc:407] Connection header received
0812 01:21:28.854640 (+ 691us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:21:28.854641 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:21:28.854654 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:21:28.854730 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:21:28.855484 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:28.856306 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:28.858041 (+ 1735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:28.858353 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:28.858393 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:28.861028 (+ 2635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:21:28.861029 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:21:28.861030 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:21:28.861037 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:21:28.863044 (+ 2007us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:21:28.863539 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:21:28.863540 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:21:28.863540 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:21:28.863598 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:21:28.863884 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:21:28.863885 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:21:28.863885 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:21:28.864024 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:21:28.864130 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0812 01:21:28.864487 (+ 357us) server_negotiation.cc:299] Negotiation successful
0812 01:21:28.864552 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":120}
I20260812 01:21:28.864738 14271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:28.853816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40050 (local address 127.24.0.190:32795)
0812 01:21:28.854919 (+ 1103us) server_negotiation.cc:206] Beginning negotiation
0812 01:21:28.854921 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:21:28.854928 (+ 7us) server_negotiation.cc:407] Connection header received
0812 01:21:28.854952 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:21:28.854953 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:21:28.854965 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:21:28.855879 (+ 914us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:21:28.855918 (+ 39us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:28.856700 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:28.857845 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:28.858151 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:28.860072 (+ 1921us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:28.860139 (+ 67us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:21:28.860140 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:21:28.860141 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:21:28.860291 (+ 150us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:21:28.863009 (+ 2718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:21:28.863478 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:21:28.863479 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:21:28.863479 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:21:28.863555 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:21:28.863788 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:21:28.863789 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:21:28.863789 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:21:28.863927 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:21:28.864488 (+ 561us) server_negotiation.cc:1035] Waiting for connection context
0812 01:21:28.864535 (+ 47us) server_negotiation.cc:299] Negotiation successful
0812 01:21:28.864643 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1081,"thread_start_us":76,"threads_started":1}
I20260812 01:21:28.865262 14270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:28.853451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:32795 (local address 127.24.0.131:54383)
0812 01:21:28.853818 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:28.853869 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 01:21:28.854497 (+ 628us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:21:28.854875 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:21:28.854879 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:21:28.855101 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:21:28.855371 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:21:28.855377 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:21:28.856662 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:21:28.856663 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:21:28.857936 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:21:28.857939 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:21:28.858504 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:21:28.858505 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:21:28.858643 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:28.859115 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:21:28.859125 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:21:28.860913 (+ 1788us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:21:28.863138 (+ 2225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:21:28.863144 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:21:28.863146 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:21:28.863444 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:21:28.863666 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:21:28.863666 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:21:28.863667 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:21:28.863801 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:21:28.864181 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:21:28.864187 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:21:28.864305 (+ 118us) client_negotiation.cc:769] Sending connection context
0812 01:21:28.864777 (+ 472us) client_negotiation.cc:240] Negotiation successful
0812 01:21:28.865037 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":102,"threads_started":1}
I20260812 01:21:28.865778 14264 heartbeater.cc:347] Connected to a master server at 127.24.0.190:32795
I20260812 01:21:28.865890 14264 heartbeater.cc:464] Registering TS with master...
I20260812 01:21:28.866091 14264 heartbeater.cc:511] Master 127.24.0.190:32795 requested a full tablet report, sending...
W20260812 01:21:28.866871 14264 heartbeater.cc:644] Failed to heartbeat to 127.24.0.190:32795 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 01:21:28.866951 14264 heartbeater.cc:464] Registering TS with master...
I20260812 01:21:28.867014 14264 heartbeater.cc:511] Master 127.24.0.190:32795 requested a full tablet report, sending...
I20260812 01:21:28.867468 14264 heartbeater.cc:464] Registering TS with master...
I20260812 01:21:28.867568 14264 heartbeater.cc:511] Master 127.24.0.190:32795 requested a full tablet report, sending...
W20260812 01:21:28.867949 14264 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 12 01:21:29 dist-test-slave-t3q3 krb5kdc[11641](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786497689, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:21:29.515795 13779 heartbeater.cc:464] Registering TS with master...
I20260812 01:21:29.515965 13779 heartbeater.cc:511] Master 127.24.0.190:32795 requested a full tablet report, sending...
I20260812 01:21:29.516423 13428 ts_manager.cc:194] Registered new tserver with Master: 1e21cae29c3949918ecc14a01ef9a798 (127.24.0.129:33711)
I20260812 01:21:29.517457 13428 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.0.129@KRBTEST.COM'} at 127.24.0.129:56899
I20260812 01:21:29.694703 14026 heartbeater.cc:464] Registering TS with master...
I20260812 01:21:29.694875 14026 heartbeater.cc:511] Master 127.24.0.190:32795 requested a full tablet report, sending...
I20260812 01:21:29.695207 13428 ts_manager.cc:194] Registered new tserver with Master: d107a76b6b9a45329560f6af8c96fe62 (127.24.0.130:39033)
I20260812 01:21:29.695763 13428 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.0.130@KRBTEST.COM'} at 127.24.0.130:51929
Aug 12 01:21:29 dist-test-slave-t3q3 krb5kdc[11641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497689, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.190@KRBTEST.COM for HTTP/127.24.0.148@KRBTEST.COM
I20260812 01:21:29.868120 14264 heartbeater.cc:464] Registering TS with master...
I20260812 01:21:29.868279 14264 heartbeater.cc:511] Master 127.24.0.190:32795 requested a full tablet report, sending...
I20260812 01:21:29.868615 13428 ts_manager.cc:194] Registered new tserver with Master: 0d72c8d421964ec1ad457c7f7b815931 (127.24.0.131:40871)
I20260812 01:21:29.869150 13428 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.0.131@KRBTEST.COM'} at 127.24.0.131:54383
I20260812 01:21:29.876047 24578 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 01:21:29.887050 14308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:29.878426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36334 (local address 127.24.0.190:32795)
0812 01:21:29.878733 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0812 01:21:29.878735 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:21:29.878846 (+ 111us) server_negotiation.cc:407] Connection header received
0812 01:21:29.878943 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:21:29.878944 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:21:29.878956 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:21:29.879001 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:21:29.879773 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:29.880579 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:29.881657 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:29.881974 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:29.882012 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:29.883853 (+ 1841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:21:29.883854 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:21:29.883854 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:21:29.883862 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:21:29.885680 (+ 1818us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:21:29.886087 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:21:29.886088 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:21:29.886088 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:21:29.886159 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:21:29.886405 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:21:29.886406 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:21:29.886406 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:21:29.886546 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:21:29.886627 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 01:21:29.886860 (+ 233us) server_negotiation.cc:299] Negotiation successful
0812 01:21:29.886949 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281,"thread_start_us":72,"threads_started":1}
I20260812 01:21:29.889685 13428 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36334:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 01:21:29.892050 13428 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 01:21:29.923941 14320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:29.909131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36630 (local address 127.24.0.130:39033)
0812 01:21:29.912202 (+ 3071us) server_negotiation.cc:206] Beginning negotiation
0812 01:21:29.912208 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 01:21:29.914971 (+ 2763us) server_negotiation.cc:407] Connection header received
0812 01:21:29.915016 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:21:29.915018 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:21:29.915036 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:21:29.915108 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:21:29.915386 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:29.916551 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:29.918111 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:29.919235 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:29.919308 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:29.919368 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 01:21:29.923802 (+ 4434us) server_negotiation.cc:299] Negotiation successful
0812 01:21:29.923849 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3027,"thread_start_us":73,"threads_started":1}
I20260812 01:21:29.924640 14321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:29.909263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.131:40871 (local address 127.0.0.1:47282)
0812 01:21:29.919433 (+ 10170us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:29.919445 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 01:21:29.919498 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:21:29.919805 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:21:29.919806 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:21:29.919808 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:21:29.919848 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:21:29.919849 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:21:29.921085 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:21:29.921087 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:21:29.922213 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:21:29.922216 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:21:29.924364 (+ 2148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:21:29.924365 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:21:29.924432 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:29.924437 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 01:21:29.924492 (+ 55us) client_negotiation.cc:240] Negotiation successful
0812 01:21:29.924554 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10149,"thread_start_us":56,"threads_started":1}
I20260812 01:21:29.925028 14327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:29.909583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47282 (local address 127.24.0.131:40871)
0812 01:21:29.914628 (+ 5045us) server_negotiation.cc:206] Beginning negotiation
0812 01:21:29.914634 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 01:21:29.919565 (+ 4931us) server_negotiation.cc:407] Connection header received
0812 01:21:29.919609 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:21:29.919612 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:21:29.919629 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:21:29.919712 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:21:29.919926 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:29.920984 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:29.922315 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:29.923083 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:29.923207 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:29.923270 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 01:21:29.924904 (+ 1634us) server_negotiation.cc:299] Negotiation successful
0812 01:21:29.924946 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5000,"thread_start_us":70,"threads_started":1}
I20260812 01:21:29.926146 14145 tablet_service.cc:1467] Processing CreateTablet for tablet 3b7f0c922a704de3ba6c16f0e2b3ae4f (DEFAULT_TABLE table=test-table [id=d31a2c6caaae459080d772a8822ce793]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:21:29.926564 14145 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b7f0c922a704de3ba6c16f0e2b3ae4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:21:29.925310 14325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:29.909030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.130:39033 (local address 127.0.0.1:36630)
0812 01:21:29.914733 (+ 5703us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:29.914796 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0812 01:21:29.914875 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:21:29.915212 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:21:29.915215 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:21:29.915218 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:21:29.915297 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:21:29.915298 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:21:29.916655 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:21:29.916656 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:21:29.918007 (+ 1351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:21:29.918011 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:21:29.923308 (+ 5297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:21:29.923309 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:21:29.923396 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:29.923409 (+ 13us) client_negotiation.cc:769] Sending connection context
0812 01:21:29.925196 (+ 1787us) client_negotiation.cc:240] Negotiation successful
0812 01:21:29.925247 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5649,"thread_start_us":94,"threads_started":1}
I20260812 01:21:29.931948 14328 tablet_bootstrap.cc:492] T 3b7f0c922a704de3ba6c16f0e2b3ae4f P 0d72c8d421964ec1ad457c7f7b815931: Bootstrap starting.
I20260812 01:21:29.932139 13904 tablet_service.cc:1467] Processing CreateTablet for tablet 3b7f0c922a704de3ba6c16f0e2b3ae4f (DEFAULT_TABLE table=test-table [id=d31a2c6caaae459080d772a8822ce793]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:21:29.932564 13904 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b7f0c922a704de3ba6c16f0e2b3ae4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:21:29.932724 14328 tablet_bootstrap.cc:654] T 3b7f0c922a704de3ba6c16f0e2b3ae4f P 0d72c8d421964ec1ad457c7f7b815931: Neither blocks nor log segments found. Creating new log.
I20260812 01:21:29.933068 14328 log.cc:824] T 3b7f0c922a704de3ba6c16f0e2b3ae4f P 0d72c8d421964ec1ad457c7f7b815931: Log is configured to *not* fsync() on all Append() calls
I20260812 01:21:29.937611 14325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:29.909394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.129:33711 (local address 127.0.0.1:36256)
0812 01:21:29.931679 (+ 22285us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:29.931691 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 01:21:29.931736 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:21:29.933598 (+ 1862us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:21:29.933599 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:21:29.933601 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:21:29.933647 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:21:29.933648 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:21:29.935082 (+ 1434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:21:29.935083 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:21:29.936274 (+ 1191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:21:29.936278 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:21:29.937390 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:21:29.937391 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:21:29.937456 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:29.937460 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 01:21:29.937492 (+ 32us) client_negotiation.cc:240] Negotiation successful
0812 01:21:29.937534 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22271,"thread_start_us":51,"threads_started":1}
I20260812 01:21:29.938570 14326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:29.909526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36256 (local address 127.24.0.129:33711)
0812 01:21:29.933399 (+ 23873us) server_negotiation.cc:206] Beginning negotiation
0812 01:21:29.933404 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 01:21:29.933418 (+ 14us) server_negotiation.cc:407] Connection header received
0812 01:21:29.933463 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:21:29.933465 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:21:29.933482 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:21:29.933715 (+ 233us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:21:29.933796 (+ 81us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:29.934926 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:29.936384 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:29.937290 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:29.938346 (+ 1056us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:29.938420 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 01:21:29.938449 (+ 29us) server_negotiation.cc:299] Negotiation successful
0812 01:21:29.938489 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23834,"thread_start_us":74,"threads_started":1}
I20260812 01:21:29.939381 13651 tablet_service.cc:1467] Processing CreateTablet for tablet 3b7f0c922a704de3ba6c16f0e2b3ae4f (DEFAULT_TABLE table=test-table [id=d31a2c6caaae459080d772a8822ce793]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:21:29.939771 13651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b7f0c922a704de3ba6c16f0e2b3ae4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:21:29.944012 14331 tablet_bootstrap.cc:492] T 3b7f0c922a704de3ba6c16f0e2b3ae4f P d107a76b6b9a45329560f6af8c96fe62: Bootstrap starting.
I20260812 01:21:29.944774 14328 tablet_bootstrap.cc:492] T 3b7f0c922a704de3ba6c16f0e2b3ae4f P 0d72c8d421964ec1ad457c7f7b815931: No bootstrap required, opened a new log
I20260812 01:21:29.944823 14331 tablet_bootstrap.cc:654] T 3b7f0c922a704de3ba6c16f0e2b3ae4f P d107a76b6b9a45329560f6af8c96fe62: Neither blocks nor log segments found. Creating new log.
I20260812 01:21:29.944918 14328 ts_tablet_manager.cc:1397] T 3b7f0c922a704de3ba6c16f0e2b3ae4f P 0d72c8d421964ec1ad457c7f7b815931: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.000s
I20260812 01:21:29.945348 14331 log.cc:824] T 3b7f0c922a704de3ba6c16f0e2b3ae4f P d107a76b6b9a45329560f6af8c96fe62: Log is configured to *not* fsync() on all Append() calls
I20260812 01:21:29.946238 14331 tablet_bootstrap.cc:492] T 3b7f0c922a704de3ba6c16f0e2b3ae4f P d107a76b6b9a45329560f6af8c96fe62: No bootstrap required, opened a new log
I20260812 01:21:29.946305 14331 ts_tablet_manager.cc:1397] T 3b7f0c922a704de3ba6c16f0e2b3ae4f P d107a76b6b9a45329560f6af8c96fe62: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 01:21:29.947717 14328 raft_consensus.cc:348] T 3b7f0c922a704de3ba6c16f0e2b3ae4f P 0d72c8d421964ec1ad457c7f7b815931 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d107a76b6b9a45329560f6af8c96fe62" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39033 } } peers { permanent_uuid: "0d72c8d421964ec1ad457c7f7b815931" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 40871 } } peers { permanent_uuid: "1e21cae29c3949918ecc14a01ef9a798" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 33711 } }
I20260812 01:21:29.947885 14328 raft_consensus.cc:374] T 3b7f0c922a704de3ba6c16f0e2b3ae4f P 0d72c8d421964ec1ad457c7f7b815931 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:21:29.947916 14328 raft_consensus.cc:729] T 3b7f0c922a704de3ba6c16f0e2b3ae4f P 0d72c8d421964ec1ad457c7f7b815931 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d72c8d421964ec1ad457c7f7b815931, State: Initialized, Role: FOLLOWER
I20260812 01:21:29.948084 14328 consensus_queue.cc:260] T 3b7f0c922a704de3ba6c16f0e2b3ae4f P 0d72c8d421964ec1ad457c7f7b815931 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d107a76b6b9a45329560f6af8c96fe62" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39033 } } peers { permanent_uuid: "0d72c8d421964ec1ad457c7f7b815931" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 40871 } } peers { permanent_uuid: "1e21cae29c3949918ecc14a01ef9a798" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 33711 } }
I20260812 01:21:29.948355 14328 ts_tablet_manager.cc:1428] T 3b7f0c922a704de3ba6c16f0e2b3ae4f P 0d72c8d421964ec1ad457c7f7b815931: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 01:21:29.948467 14264 heartbeater.cc:503] Master 127.24.0.190:32795 was elected leader, sending a full tablet report...
I20260812 01:21:29.948969 14331 raft_consensus.cc:348] T 3b7f0c922a704de3ba6c16f0e2b3ae4f P d107a76b6b9a45329560f6af8c96fe62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d107a76b6b9a45329560f6af8c96fe62" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39033 } } peers { permanent_uuid: "0d72c8d421964ec1ad457c7f7b815931" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 40871 } } peers { permanent_uuid: "1e21cae29c3949918ecc14a01ef9a798" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 33711 } }
I20260812 01:21:29.949141 14331 raft_consensus.cc:374] T 3b7f0c922a704de3ba6c16f0e2b3ae4f P d107a76b6b9a45329560f6af8c96fe62 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:21:29.949174 14331 raft_consensus.cc:729] T 3b7f0c922a704de3ba6c16f0e2b3ae4f P d107a76b6b9a45329560f6af8c96fe62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d107a76b6b9a45329560f6af8c96fe62, State: Initialized, Role: FOLLOWER
I20260812 01:21:29.949283 14331 consensus_queue.cc:260] T 3b7f0c922a704de3ba6c16f0e2b3ae4f P d107a76b6b9a45329560f6af8c96fe62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d107a76b6b9a45329560f6af8c96fe62" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39033 } } peers { permanent_uuid: "0d72c8d421964ec1ad457c7f7b815931" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 40871 } } peers { permanent_uuid: "1e21cae29c3949918ecc14a01ef9a798" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 33711 } }
I20260812 01:21:29.949536 14331 ts_tablet_manager.cc:1428] T 3b7f0c922a704de3ba6c16f0e2b3ae4f P d107a76b6b9a45329560f6af8c96fe62: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 01:21:29.949695 14332 tablet_bootstrap.cc:492] T 3b7f0c922a704de3ba6c16f0e2b3ae4f P 1e21cae29c3949918ecc14a01ef9a798: Bootstrap starting.
I20260812 01:21:29.950330 14026 heartbeater.cc:503] Master 127.24.0.190:32795 was elected leader, sending a full tablet report...
I20260812 01:21:29.950429 14332 tablet_bootstrap.cc:654] T 3b7f0c922a704de3ba6c16f0e2b3ae4f P 1e21cae29c3949918ecc14a01ef9a798: Neither blocks nor log segments found. Creating new log.
I20260812 01:21:29.950765 14332 log.cc:824] T 3b7f0c922a704de3ba6c16f0e2b3ae4f P 1e21cae29c3949918ecc14a01ef9a798: Log is configured to *not* fsync() on all Append() calls
I20260812 01:21:29.952544 14332 tablet_bootstrap.cc:492] T 3b7f0c922a704de3ba6c16f0e2b3ae4f P 1e21cae29c3949918ecc14a01ef9a798: No bootstrap required, opened a new log
I20260812 01:21:29.952668 14332 ts_tablet_manager.cc:1397] T 3b7f0c922a704de3ba6c16f0e2b3ae4f P 1e21cae29c3949918ecc14a01ef9a798: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 01:21:29.955338 14332 raft_consensus.cc:348] T 3b7f0c922a704de3ba6c16f0e2b3ae4f P 1e21cae29c3949918ecc14a01ef9a798 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d107a76b6b9a45329560f6af8c96fe62" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39033 } } peers { permanent_uuid: "0d72c8d421964ec1ad457c7f7b815931" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 40871 } } peers { permanent_uuid: "1e21cae29c3949918ecc14a01ef9a798" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 33711 } }
I20260812 01:21:29.955660 14332 raft_consensus.cc:374] T 3b7f0c922a704de3ba6c16f0e2b3ae4f P 1e21cae29c3949918ecc14a01ef9a798 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:21:29.955757 14332 raft_consensus.cc:729] T 3b7f0c922a704de3ba6c16f0e2b3ae4f P 1e21cae29c3949918ecc14a01ef9a798 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e21cae29c3949918ecc14a01ef9a798, State: Initialized, Role: FOLLOWER
I20260812 01:21:29.955962 14332 consensus_queue.cc:260] T 3b7f0c922a704de3ba6c16f0e2b3ae4f P 1e21cae29c3949918ecc14a01ef9a798 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d107a76b6b9a45329560f6af8c96fe62" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39033 } } peers { permanent_uuid: "0d72c8d421964ec1ad457c7f7b815931" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 40871 } } peers { permanent_uuid: "1e21cae29c3949918ecc14a01ef9a798" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 33711 } }
I20260812 01:21:29.956351 14332 ts_tablet_manager.cc:1428] T 3b7f0c922a704de3ba6c16f0e2b3ae4f P 1e21cae29c3949918ecc14a01ef9a798: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260812 01:21:29.956511 13779 heartbeater.cc:503] Master 127.24.0.190:32795 was elected leader, sending a full tablet report...
W20260812 01:21:29.998284 13780 tablet.cc:2369] T 3b7f0c922a704de3ba6c16f0e2b3ae4f P 1e21cae29c3949918ecc14a01ef9a798: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 01:21:30.038532 14343 raft_consensus.cc:482] T 3b7f0c922a704de3ba6c16f0e2b3ae4f P 1e21cae29c3949918ecc14a01ef9a798 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 01:21:30.038702 14343 raft_consensus.cc:504] T 3b7f0c922a704de3ba6c16f0e2b3ae4f P 1e21cae29c3949918ecc14a01ef9a798 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d107a76b6b9a45329560f6af8c96fe62" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39033 } } peers { permanent_uuid: "0d72c8d421964ec1ad457c7f7b815931" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 40871 } } peers { permanent_uuid: "1e21cae29c3949918ecc14a01ef9a798" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 33711 } }
I20260812 01:21:30.039124 14343 leader_election.cc:288] T 3b7f0c922a704de3ba6c16f0e2b3ae4f P 1e21cae29c3949918ecc14a01ef9a798 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d107a76b6b9a45329560f6af8c96fe62 (127.24.0.130:39033), 0d72c8d421964ec1ad457c7f7b815931 (127.24.0.131:40871)
I20260812 01:21:30.043514 14320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:30.039425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.129:49515 (local address 127.24.0.130:39033)
0812 01:21:30.039485 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0812 01:21:30.039488 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:21:30.039931 (+ 443us) server_negotiation.cc:407] Connection header received
0812 01:21:30.040045 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:21:30.040046 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:21:30.040076 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:21:30.040108 (+ 32us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:21:30.040348 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:30.041060 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:30.042616 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:30.043140 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:30.043185 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:30.043194 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 01:21:30.043405 (+ 211us) server_negotiation.cc:299] Negotiation successful
0812 01:21:30.043442 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260812 01:21:30.043614 14327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:30.039727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.129:44961 (local address 127.24.0.131:40871)
0812 01:21:30.039766 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0812 01:21:30.039768 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:21:30.039923 (+ 155us) server_negotiation.cc:407] Connection header received
0812 01:21:30.040044 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:21:30.040046 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:21:30.040075 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:21:30.040108 (+ 33us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:21:30.040453 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:30.041167 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:30.042617 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:30.043185 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:30.043237 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:30.043246 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 01:21:30.043492 (+ 246us) server_negotiation.cc:299] Negotiation successful
0812 01:21:30.043538 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 01:21:30.043900 14344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:30.039646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.131:40871 (local address 127.24.0.129:44961)
0812 01:21:30.039850 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:30.039861 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 01:21:30.039959 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:21:30.040183 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:21:30.040183 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:21:30.040185 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:21:30.040284 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:21:30.040286 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:21:30.041277 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:21:30.041278 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:21:30.042502 (+ 1224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:21:30.042506 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:21:30.043310 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:21:30.043310 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:21:30.043389 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:30.043393 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 01:21:30.043756 (+ 363us) client_negotiation.cc:240] Negotiation successful
0812 01:21:30.043801 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"mutex_wait_us":4,"thread_start_us":120,"threads_started":1}
I20260812 01:21:30.044135 14345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:30.039351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.130:39033 (local address 127.24.0.129:49515)
0812 01:21:30.039848 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:30.039859 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 01:21:30.039959 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:21:30.040179 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:21:30.040180 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:21:30.040181 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:21:30.040270 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:21:30.040272 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:21:30.041163 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:21:30.041164 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:21:30.042498 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:21:30.042502 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:21:30.043268 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:21:30.043269 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:21:30.043330 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:30.043335 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 01:21:30.043896 (+ 561us) client_negotiation.cc:240] Negotiation successful
0812 01:21:30.043943 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"mutex_wait_us":6,"thread_start_us":174,"threads_started":1}
I20260812 01:21:30.044376 14190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b7f0c922a704de3ba6c16f0e2b3ae4f" candidate_uuid: "1e21cae29c3949918ecc14a01ef9a798" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d72c8d421964ec1ad457c7f7b815931" is_pre_election: true
I20260812 01:21:30.044600 14190 raft_consensus.cc:2454] T 3b7f0c922a704de3ba6c16f0e2b3ae4f P 0d72c8d421964ec1ad457c7f7b815931 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1e21cae29c3949918ecc14a01ef9a798 in term 0.
I20260812 01:21:30.044672 13947 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b7f0c922a704de3ba6c16f0e2b3ae4f" candidate_uuid: "1e21cae29c3949918ecc14a01ef9a798" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d107a76b6b9a45329560f6af8c96fe62" is_pre_election: true
I20260812 01:21:30.044843 13947 raft_consensus.cc:2454] T 3b7f0c922a704de3ba6c16f0e2b3ae4f P d107a76b6b9a45329560f6af8c96fe62 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1e21cae29c3949918ecc14a01ef9a798 in term 0.
I20260812 01:21:30.044858 13580 leader_election.cc:302] T 3b7f0c922a704de3ba6c16f0e2b3ae4f P 1e21cae29c3949918ecc14a01ef9a798 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d72c8d421964ec1ad457c7f7b815931, 1e21cae29c3949918ecc14a01ef9a798; no voters:
I20260812 01:21:30.044984 14343 raft_consensus.cc:2781] T 3b7f0c922a704de3ba6c16f0e2b3ae4f P 1e21cae29c3949918ecc14a01ef9a798 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 01:21:30.045038 14343 raft_consensus.cc:482] T 3b7f0c922a704de3ba6c16f0e2b3ae4f P 1e21cae29c3949918ecc14a01ef9a798 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 01:21:30.045066 14343 raft_consensus.cc:3037] T 3b7f0c922a704de3ba6c16f0e2b3ae4f P 1e21cae29c3949918ecc14a01ef9a798 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:21:30.046111 14343 raft_consensus.cc:504] T 3b7f0c922a704de3ba6c16f0e2b3ae4f P 1e21cae29c3949918ecc14a01ef9a798 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d107a76b6b9a45329560f6af8c96fe62" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39033 } } peers { permanent_uuid: "0d72c8d421964ec1ad457c7f7b815931" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 40871 } } peers { permanent_uuid: "1e21cae29c3949918ecc14a01ef9a798" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 33711 } }
I20260812 01:21:30.046262 14343 leader_election.cc:288] T 3b7f0c922a704de3ba6c16f0e2b3ae4f P 1e21cae29c3949918ecc14a01ef9a798 [CANDIDATE]: Term 1 election: Requested vote from peers d107a76b6b9a45329560f6af8c96fe62 (127.24.0.130:39033), 0d72c8d421964ec1ad457c7f7b815931 (127.24.0.131:40871)
I20260812 01:21:30.046536 13947 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b7f0c922a704de3ba6c16f0e2b3ae4f" candidate_uuid: "1e21cae29c3949918ecc14a01ef9a798" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d107a76b6b9a45329560f6af8c96fe62"
I20260812 01:21:30.046515 14190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b7f0c922a704de3ba6c16f0e2b3ae4f" candidate_uuid: "1e21cae29c3949918ecc14a01ef9a798" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d72c8d421964ec1ad457c7f7b815931"
I20260812 01:21:30.046613 13947 raft_consensus.cc:3037] T 3b7f0c922a704de3ba6c16f0e2b3ae4f P d107a76b6b9a45329560f6af8c96fe62 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:21:30.046622 14190 raft_consensus.cc:3037] T 3b7f0c922a704de3ba6c16f0e2b3ae4f P 0d72c8d421964ec1ad457c7f7b815931 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:21:30.047524 13947 raft_consensus.cc:2454] T 3b7f0c922a704de3ba6c16f0e2b3ae4f P d107a76b6b9a45329560f6af8c96fe62 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1e21cae29c3949918ecc14a01ef9a798 in term 1.
I20260812 01:21:30.047580 14190 raft_consensus.cc:2454] T 3b7f0c922a704de3ba6c16f0e2b3ae4f P 0d72c8d421964ec1ad457c7f7b815931 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1e21cae29c3949918ecc14a01ef9a798 in term 1.
I20260812 01:21:30.047787 13580 leader_election.cc:302] T 3b7f0c922a704de3ba6c16f0e2b3ae4f P 1e21cae29c3949918ecc14a01ef9a798 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d72c8d421964ec1ad457c7f7b815931, 1e21cae29c3949918ecc14a01ef9a798; no voters:
I20260812 01:21:30.047902 14343 raft_consensus.cc:2781] T 3b7f0c922a704de3ba6c16f0e2b3ae4f P 1e21cae29c3949918ecc14a01ef9a798 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:21:30.048112 14343 raft_consensus.cc:686] T 3b7f0c922a704de3ba6c16f0e2b3ae4f P 1e21cae29c3949918ecc14a01ef9a798 [term 1 LEADER]: Becoming Leader. State: Replica: 1e21cae29c3949918ecc14a01ef9a798, State: Running, Role: LEADER
I20260812 01:21:30.048213 14343 consensus_queue.cc:237] T 3b7f0c922a704de3ba6c16f0e2b3ae4f P 1e21cae29c3949918ecc14a01ef9a798 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d107a76b6b9a45329560f6af8c96fe62" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39033 } } peers { permanent_uuid: "0d72c8d421964ec1ad457c7f7b815931" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 40871 } } peers { permanent_uuid: "1e21cae29c3949918ecc14a01ef9a798" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 33711 } }
I20260812 01:21:30.049018 13428 catalog_manager.cc:5495] T 3b7f0c922a704de3ba6c16f0e2b3ae4f P 1e21cae29c3949918ecc14a01ef9a798 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1e21cae29c3949918ecc14a01ef9a798 (127.24.0.129). New cstate: current_term: 1 leader_uuid: "1e21cae29c3949918ecc14a01ef9a798" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d107a76b6b9a45329560f6af8c96fe62" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d72c8d421964ec1ad457c7f7b815931" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 40871 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1e21cae29c3949918ecc14a01ef9a798" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 33711 } health_report { overall_health: HEALTHY } } }
I20260812 01:21:30.090955 14326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:30.087566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36268 (local address 127.24.0.129:33711)
0812 01:21:30.087613 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 01:21:30.087615 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:21:30.087625 (+ 10us) server_negotiation.cc:407] Connection header received
0812 01:21:30.087677 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:21:30.087678 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:21:30.087717 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:21:30.087811 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:21:30.087997 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:30.088775 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:30.089912 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:30.090211 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:30.090289 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:30.090535 (+ 246us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:21:30.090679 (+ 144us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:21:30.090793 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0812 01:21:30.090841 (+ 48us) server_negotiation.cc:299] Negotiation successful
0812 01:21:30.090868 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 01:21:30.093187 13947 raft_consensus.cc:1261] T 3b7f0c922a704de3ba6c16f0e2b3ae4f P d107a76b6b9a45329560f6af8c96fe62 [term 1 FOLLOWER]: Refusing update from remote peer 1e21cae29c3949918ecc14a01ef9a798: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:21:30.093189 14190 raft_consensus.cc:1261] T 3b7f0c922a704de3ba6c16f0e2b3ae4f P 0d72c8d421964ec1ad457c7f7b815931 [term 1 FOLLOWER]: Refusing update from remote peer 1e21cae29c3949918ecc14a01ef9a798: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:21:30.094326 14343 consensus_queue.cc:1035] T 3b7f0c922a704de3ba6c16f0e2b3ae4f P 1e21cae29c3949918ecc14a01ef9a798 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d107a76b6b9a45329560f6af8c96fe62" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:21:30.094473 14346 consensus_queue.cc:1035] T 3b7f0c922a704de3ba6c16f0e2b3ae4f P 1e21cae29c3949918ecc14a01ef9a798 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d72c8d421964ec1ad457c7f7b815931" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 40871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:21:30.097569 14355 mvcc.cc:205] Tried to move back new op lower bound from 7317494538618458112 to 7317494538438037504. Current Snapshot: MvccSnapshot[applied={T|T < 7317494538618458112}]
I20260812 01:21:30.097823 14349 mvcc.cc:205] Tried to move back new op lower bound from 7317494538618458112 to 7317494538438037504. Current Snapshot: MvccSnapshot[applied={T|T < 7317494538618458112}]
I20260812 01:21:30.100746 13427 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36334:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:21:30.100839 13427 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36334:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:21:30.101657 13427 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 19c171e19e4544058f8163e975c9dfa3: Sending DeleteTablet for 3 replicas of tablet 3b7f0c922a704de3ba6c16f0e2b3ae4f
I20260812 01:21:30.102145 13904 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b7f0c922a704de3ba6c16f0e2b3ae4f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:21:30 UTC) from {username='kudu', principal='kudu/127.24.0.190@KRBTEST.COM'} at 127.0.0.1:36630
I20260812 01:21:30.102381 14145 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b7f0c922a704de3ba6c16f0e2b3ae4f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:21:30 UTC) from {username='kudu', principal='kudu/127.24.0.190@KRBTEST.COM'} at 127.0.0.1:47282
I20260812 01:21:30.102566 13651 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b7f0c922a704de3ba6c16f0e2b3ae4f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:21:30 UTC) from {username='kudu', principal='kudu/127.24.0.190@KRBTEST.COM'} at 127.0.0.1:36256
I20260812 01:21:30.103250 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 13493
W20260812 01:21:30.107005 14265 tablet.cc:2369] T 3b7f0c922a704de3ba6c16f0e2b3ae4f P 0d72c8d421964ec1ad457c7f7b815931: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 01:21:30.116230 14365 tablet_replica.cc:330] T 3b7f0c922a704de3ba6c16f0e2b3ae4f P d107a76b6b9a45329560f6af8c96fe62: stopping tablet replica
I20260812 01:21:30.116389 14365 raft_consensus.cc:2229] T 3b7f0c922a704de3ba6c16f0e2b3ae4f P d107a76b6b9a45329560f6af8c96fe62 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 01:21:30.116539 14365 raft_consensus.cc:2258] T 3b7f0c922a704de3ba6c16f0e2b3ae4f P d107a76b6b9a45329560f6af8c96fe62 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:21:30.116762 14366 tablet_replica.cc:330] T 3b7f0c922a704de3ba6c16f0e2b3ae4f P 0d72c8d421964ec1ad457c7f7b815931: stopping tablet replica
I20260812 01:21:30.116889 14366 raft_consensus.cc:2229] T 3b7f0c922a704de3ba6c16f0e2b3ae4f P 0d72c8d421964ec1ad457c7f7b815931 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 01:21:30.116966 14366 pending_rounds.cc:62] T 3b7f0c922a704de3ba6c16f0e2b3ae4f P 0d72c8d421964ec1ad457c7f7b815931: Trying to abort 2 pending ops.
I20260812 01:21:30.117005 14366 pending_rounds.cc:68] T 3b7f0c922a704de3ba6c16f0e2b3ae4f P 0d72c8d421964ec1ad457c7f7b815931: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7317494538618458112 op_type: WRITE_OP write_request { tablet_id: "3b7f0c922a704de3ba6c16f0e2b3ae4f" schema { columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } columns { name: "val" type: INT32 is_key: false is_nullable: true immutable: false } } row_operations { rows: "<redacted>""\001\003\00090\000\0001\324\000\000" indirect_data: "<redacted>""" } external_consistency_mode: CLIENT_PROPAGATED authz_token { token_data: "\010\306\227\357\323\006\"8\n\ntest-admin\022*\n d31a2c6caaae459080d772a8822ce793\020\001\030\001 \001(\001" signature: "<redacted>""\212\2733\356A\n\264-\260\003w\317\021\274\377p\227\347uO!m\372\031E\223\030\216r\007\341\016\26685b\327?\232WCf\t\354\331\'\013\034\035\366#\367\334\035\374\344W> \031_\370\031c" signing_key_seq_num: 0 } } request_id { client_id: "a593551872a049d0bbd086edf71b7078" seq_no: 0 first_incomplete_seq_no: 0 attempt_no: 0 }
I20260812 01:21:30.117264 14366 pending_rounds.cc:68] T 3b7f0c922a704de3ba6c16f0e2b3ae4f P 0d72c8d421964ec1ad457c7f7b815931: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7317494538438037504 op_type: NO_OP noop_request { }
I20260812 01:21:30.117322 14366 raft_consensus.cc:2866] T 3b7f0c922a704de3ba6c16f0e2b3ae4f P 0d72c8d421964ec1ad457c7f7b815931 [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260812 01:21:30.117358 14366 raft_consensus.cc:2258] T 3b7f0c922a704de3ba6c16f0e2b3ae4f P 0d72c8d421964ec1ad457c7f7b815931 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:21:30.117547 14365 ts_tablet_manager.cc:1905] T 3b7f0c922a704de3ba6c16f0e2b3ae4f P d107a76b6b9a45329560f6af8c96fe62: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260812 01:21:30.122834 14062 connection.cc:535] server connection from 127.24.0.129:44961 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 01:21:30.123466 14366 ts_tablet_manager.cc:1905] T 3b7f0c922a704de3ba6c16f0e2b3ae4f P 0d72c8d421964ec1ad457c7f7b815931: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260812 01:21:30.125681 13393 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.0.129:33711 (error 108)
I20260812 01:21:30.125866 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 13784
I20260812 01:21:30.138214 14366 ts_tablet_manager.cc:1918] T 3b7f0c922a704de3ba6c16f0e2b3ae4f P 0d72c8d421964ec1ad457c7f7b815931: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 01:21:30.138484 14366 log.cc:1196] T 3b7f0c922a704de3ba6c16f0e2b3ae4f P 0d72c8d421964ec1ad457c7f7b815931: Deleting WAL directory at /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786497534195564-24578-0/minicluster-data/ts-2/wal/wals/3b7f0c922a704de3ba6c16f0e2b3ae4f
I20260812 01:21:30.139030 14366 ts_tablet_manager.cc:1939] T 3b7f0c922a704de3ba6c16f0e2b3ae4f P 0d72c8d421964ec1ad457c7f7b815931: Deleting consensus metadata
I20260812 01:21:30.139860 13393 catalog_manager.cc:4841] TS 0d72c8d421964ec1ad457c7f7b815931 (127.24.0.131:40871): tablet 3b7f0c922a704de3ba6c16f0e2b3ae4f (table test-table [id=d31a2c6caaae459080d772a8822ce793]) successfully deleted
I20260812 01:21:30.143893 14325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:30.143715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.129:33711 (local address 127.0.0.1:36278)
0812 01:21:30.143777 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:30.143841 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.129:33711: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260812 01:21:30.144047 13393 catalog_manager.cc:4584] TS 1e21cae29c3949918ecc14a01ef9a798 (127.24.0.129:33711): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3b7f0c922a704de3ba6c16f0e2b3ae4f: Network error: Client connection negotiation failed: client connection to 127.24.0.129:33711: connect: Connection refused (error 111)
I20260812 01:21:30.153302 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 14031
I20260812 01:21:30.153409 14325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:30.153291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.130:39033 (local address 127.0.0.1:36640)
0812 01:21:30.153344 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:30.153368 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.130:39033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260812 01:21:30.166162 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 13365
I20260812 01:21:30.180171 24578 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260812 01:21:30.526618 24578 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260812 01:21:30.526746 24578 mini_ranger.cc:66] Stopping Ranger...
I20260812 01:21:30.881232 24578 mini_ranger.cc:68] Stopped Ranger
2026-08-12 01:21:30.881 UTC [11687] LOG: received smart shutdown request
2026-08-12 01:21:30.885 UTC [11687] LOG: background worker "logical replication launcher" (PID 11698) exited with exit code 1
2026-08-12 01:21:30.886 UTC [11692] LOG: shutting down
2026-08-12 01:21:31.039 UTC [11687] LOG: database system is shut down
2026-08-12T01:21:31Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (48236 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786497534195564-24578-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 01:21:31 dist-test-slave-t3q3 krb5kdc[14386](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 01:21:31 dist-test-slave-t3q3 krb5kdc[14386](info): set up 2 sockets
Aug 12 01:21:31 dist-test-slave-t3q3 krb5kdc[14386](info): commencing operation
krb5kdc: starting...
W20260812 01:21:32.413321 24578 mini_kdc.cc:121] Time spent starting KDC: real 1.239s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786497534195564-24578-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786497534195564-24578-0/krb5kdc/test-user.keytab.
Aug 12 01:21:32 dist-test-slave-t3q3 krb5kdc[14386](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497692, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T01:21:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:21:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.0.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.0.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.0.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab.
Entry for principal kudu/127.24.0.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab.
I20260812 01:21:32.533869 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786497534195564-24578-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786497534195564-24578-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786497534195564-24578-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.190:36985
--webserver_interface=127.24.0.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab
--principal=kudu/127.24.0.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:35795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.0.190:36985,127.24.0.189:38889,127.24.0.188:33327
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:21:32.615268 14409 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:21:32.615468 14409 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:21:32.615490 14409 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:21:32.616690 14409 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:21:32.616734 14409 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:21:32.616748 14409 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:21:32.616761 14409 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:21:32.616775 14409 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:21:32.618175 14409 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:35795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786497534195564-24578-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786497534195564-24578-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.0.190:36985,127.24.0.189:38889,127.24.0.188:33327
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab
--principal=kudu/127.24.0.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.190:36985
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.0.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14409
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786497534195564-24578-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:21:32.618424 14409 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:21:32.618652 14409 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:21:32.623414 14418 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:21:32.623548 14417 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:21:32.623433 14422 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:21:32.623662 14409 server_base.cc:1034] running on GCE node
I20260812 01:21:32.623930 14409 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:21:32.624279 14409 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:21:32.625456 14409 hybrid_clock.cc:648] HybridClock initialized: now 1786497692625450 us; error 37 us; skew 500 ppm
Aug 12 01:21:32 dist-test-slave-t3q3 krb5kdc[14386](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497692, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:21:32.627815 14409 init.cc:377] Logged in from keytab as kudu/127.24.0.190@KRBTEST.COM (short username kudu)
I20260812 01:21:32.628290 14409 webserver.cc:460] Webserver started at http://127.24.0.190:33551/ using document root <none> and password file <none>
I20260812 01:21:32.628511 14409 fs_manager.cc:359] Metadata directory not provided
I20260812 01:21:32.628579 14409 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:21:32.628739 14409 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:21:32.629767 14409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786497534195564-24578-0/minicluster-data/master-0/data/instance:
uuid: "ac1fb52d504340dfa69635fd8a594ef0"
format_stamp: "Formatted at 2026-08-12 01:21:32 on dist-test-slave-t3q3"
server_key: "6b6e14e9ffa046665142ce94a292810b"
server_key_iv: "722c03500e2edb8c678979f300e25f1a"
server_key_version: "encryptionkey@0"
I20260812 01:21:32.630132 14409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786497534195564-24578-0/minicluster-data/master-0/wal/instance:
uuid: "ac1fb52d504340dfa69635fd8a594ef0"
format_stamp: "Formatted at 2026-08-12 01:21:32 on dist-test-slave-t3q3"
server_key: "6b6e14e9ffa046665142ce94a292810b"
server_key_iv: "722c03500e2edb8c678979f300e25f1a"
server_key_version: "encryptionkey@0"
I20260812 01:21:32.631438 14409 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 01:21:32.632176 14435 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:21:32.632354 14409 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 01:21:32.632457 14409 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786497534195564-24578-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786497534195564-24578-0/minicluster-data/master-0/wal
uuid: "ac1fb52d504340dfa69635fd8a594ef0"
format_stamp: "Formatted at 2026-08-12 01:21:32 on dist-test-slave-t3q3"
server_key: "6b6e14e9ffa046665142ce94a292810b"
server_key_iv: "722c03500e2edb8c678979f300e25f1a"
server_key_version: "encryptionkey@0"
I20260812 01:21:32.632544 14409 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786497534195564-24578-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786497534195564-24578-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786497534195564-24578-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:21:32.661449 14409 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:21:32.663365 14409 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:21:32.663532 14409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:21:32.667568 14409 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.190:36985
I20260812 01:21:32.667629 14528 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.190:36985 every 8 connection(s)
I20260812 01:21:32.668000 14409 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
I20260812 01:21:32.668934 14530 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:21:32.669103 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 14409
I20260812 01:21:32.669261 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786497534195564-24578-0/minicluster-data/master-0/wal/instance
I20260812 01:21:32.669472 24578 external_mini_cluster.cc:1428] Setting key 41443ec3d58a6c4c7b68e4be88b8ab21
I20260812 01:21:32.670675 14530 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.0.190" port: 36985 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 01:21:32 dist-test-slave-t3q3 krb5kdc[14386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497692, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.190@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.24.0.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.0.189@KRBTEST.COM" created.
I20260812 01:21:32.681506 14537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:32.671486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55448 (local address 127.24.0.190:36985)
0812 01:21:32.671656 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0812 01:21:32.671665 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 01:21:32.671791 (+ 126us) server_negotiation.cc:407] Connection header received
0812 01:21:32.672536 (+ 745us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:21:32.672538 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:21:32.672555 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:21:32.672631 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:21:32.673317 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:32.674718 (+ 1401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:32.675586 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:32.676171 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:32.676291 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:32.678533 (+ 2242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:21:32.678543 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:21:32.678543 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:21:32.678551 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:21:32.679975 (+ 1424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:21:32.680426 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:21:32.680429 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:21:32.680429 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:21:32.680504 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:21:32.680725 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:21:32.680726 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:21:32.680726 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:21:32.680937 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:21:32.681015 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 01:21:32.681230 (+ 215us) server_negotiation.cc:299] Negotiation successful
0812 01:21:32.681406 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":60,"server-negotiator.queue_time_us":139,"thread_start_us":76,"threads_started":1}
I20260812 01:21:32.681514 14534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:32.671457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:36985 (local address 127.0.0.1:55448)
0812 01:21:32.671645 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:32.671690 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 01:21:32.672384 (+ 694us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:21:32.672672 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:21:32.672675 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:21:32.673042 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:21:32.673205 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:21:32.673210 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:21:32.674812 (+ 1602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:21:32.674813 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:21:32.675496 (+ 683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:21:32.675498 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:21:32.676258 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:21:32.676259 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:21:32.676317 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:32.676833 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:21:32.676841 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:21:32.678438 (+ 1597us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:21:32.680050 (+ 1612us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:21:32.680056 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:21:32.680088 (+ 32us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:21:32.680289 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:21:32.680567 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:21:32.680567 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:21:32.680568 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:21:32.680660 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:21:32.681013 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:21:32.681018 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:21:32.681097 (+ 79us) client_negotiation.cc:769] Sending connection context
0812 01:21:32.681189 (+ 92us) client_negotiation.cc:240] Negotiation successful
0812 01:21:32.681312 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":90,"threads_started":1}
I20260812 01:21:32.682322 14530 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.0.189" port: 38889 } has no permanent_uuid. Determining permanent_uuid...
I20260812 01:21:32.682777 14534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:32.682643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.189:38889 (local address 127.0.0.1:39366)
0812 01:21:32.682692 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:32.682745 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.189:38889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260812 01:21:32.682914 14438 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.0.189:38889: connect: Connection refused (error 111)
I20260812 01:21:32.683362 14534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:32.683274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.189:38889 (local address 127.0.0.1:39368)
0812 01:21:32.683320 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:32.683345 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.189:38889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260812 01:21:32.683465 14530 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.0.189:38889: Network error: Client connection negotiation failed: client connection to 127.24.0.189:38889: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.0.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.189.keytab.
Entry for principal kudu/127.24.0.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.189.keytab.
I20260812 01:21:32.695585 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786497534195564-24578-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786497534195564-24578-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786497534195564-24578-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786497534195564-24578-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.189:38889
--webserver_interface=127.24.0.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.189.keytab
--principal=kudu/127.24.0.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:35795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.0.190:36985,127.24.0.189:38889,127.24.0.188:33327
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 01:21:32.736200 14530 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.0.189" port: 38889 } attempt: 1
I20260812 01:21:32.738315 14534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:32.738143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.189:38889 (local address 127.0.0.1:39372)
0812 01:21:32.738236 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:32.738275 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.189:38889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260812 01:21:32.738739 14534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:32.738658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.189:38889 (local address 127.0.0.1:39384)
0812 01:21:32.738693 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:32.738715 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.189:38889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260812 01:21:32.738818 14530 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.0.189:38889: Network error: Client connection negotiation failed: client connection to 127.24.0.189:38889: connect: Connection refused (error 111)
W20260812 01:21:32.779956 14546 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:21:32.780198 14546 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:21:32.780237 14546 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:21:32.781320 14546 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:21:32.781383 14546 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:21:32.781409 14546 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:21:32.781432 14546 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:21:32.781461 14546 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:21:32.782903 14546 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:35795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786497534195564-24578-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786497534195564-24578-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.24.0.190:36985,127.24.0.189:38889,127.24.0.188:33327
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.189.keytab
--principal=kudu/127.24.0.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.189:38889
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786497534195564-24578-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.24.0.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14546
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786497534195564-24578-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:21:32.783175 14546 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:21:32.783396 14546 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 01:21:32.788318 14546 server_base.cc:1034] running on GCE node
W20260812 01:21:32.788270 14553 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:21:32.788270 14552 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:21:32.788268 14555 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:21:32.788825 14546 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:21:32.789121 14546 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:21:32.790283 14546 hybrid_clock.cc:648] HybridClock initialized: now 1786497692790276 us; error 44 us; skew 500 ppm
Aug 12 01:21:32 dist-test-slave-t3q3 krb5kdc[14386](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497692, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:21:32.792855 14546 init.cc:377] Logged in from keytab as kudu/127.24.0.189@KRBTEST.COM (short username kudu)
I20260812 01:21:32.793287 14546 webserver.cc:460] Webserver started at http://127.24.0.189:45155/ using document root <none> and password file <none>
I20260812 01:21:32.793520 14546 fs_manager.cc:359] Metadata directory not provided
I20260812 01:21:32.793589 14546 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:21:32.793747 14546 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:21:32.794786 14546 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786497534195564-24578-0/minicluster-data/master-1/data/instance:
uuid: "0435323d5c1c411aa9c9163a986d5d0f"
format_stamp: "Formatted at 2026-08-12 01:21:32 on dist-test-slave-t3q3"
server_key: "9e231542e3a01975bd2a14ab87f5cee9"
server_key_iv: "59961e43f43e560b4b90eb87053e079e"
server_key_version: "encryptionkey@0"
I20260812 01:21:32.795158 14546 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786497534195564-24578-0/minicluster-data/master-1/wal/instance:
uuid: "0435323d5c1c411aa9c9163a986d5d0f"
format_stamp: "Formatted at 2026-08-12 01:21:32 on dist-test-slave-t3q3"
server_key: "9e231542e3a01975bd2a14ab87f5cee9"
server_key_iv: "59961e43f43e560b4b90eb87053e079e"
server_key_version: "encryptionkey@0"
I20260812 01:21:32.796526 14546 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 01:21:32.797230 14566 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:21:32.797400 14546 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 01:21:32.797504 14546 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786497534195564-24578-0/minicluster-data/master-1/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786497534195564-24578-0/minicluster-data/master-1/wal
uuid: "0435323d5c1c411aa9c9163a986d5d0f"
format_stamp: "Formatted at 2026-08-12 01:21:32 on dist-test-slave-t3q3"
server_key: "9e231542e3a01975bd2a14ab87f5cee9"
server_key_iv: "59961e43f43e560b4b90eb87053e079e"
server_key_version: "encryptionkey@0"
I20260812 01:21:32.797577 14546 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786497534195564-24578-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786497534195564-24578-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786497534195564-24578-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:21:32.806962 14530 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.0.189" port: 38889 } attempt: 2
I20260812 01:21:32.807432 14534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:32.807293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.189:38889 (local address 127.0.0.1:39386)
0812 01:21:32.807357 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:32.807401 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.189:38889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260812 01:21:32.807760 14534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:32.807685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.189:38889 (local address 127.0.0.1:39400)
0812 01:21:32.807721 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:32.807743 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.189:38889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260812 01:21:32.807873 14530 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.0.189:38889: Network error: Client connection negotiation failed: client connection to 127.24.0.189:38889: connect: Connection refused (error 111)
I20260812 01:21:32.818610 14546 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:21:32.820676 14546 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:21:32.820838 14546 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:21:32.824826 14546 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.189:38889
I20260812 01:21:32.824863 14654 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.189:38889 every 8 connection(s)
I20260812 01:21:32.825264 14546 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786497534195564-24578-0/minicluster-data/master-1/data/info.pb
I20260812 01:21:32.826169 14656 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:21:32.827737 14656 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.0.190" port: 36985 } has no permanent_uuid. Determining permanent_uuid...
I20260812 01:21:32.830482 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 14546
I20260812 01:21:32.830591 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786497534195564-24578-0/minicluster-data/master-1/wal/instance
I20260812 01:21:32.830809 24578 external_mini_cluster.cc:1428] Setting key b4093f68c98a335f97003e81addfe4c3
Aug 12 01:21:32 dist-test-slave-t3q3 krb5kdc[14386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497692, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.189@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
I20260812 01:21:32.837047 14537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:32.828476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55458 (local address 127.24.0.190:36985)
0812 01:21:32.828557 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0812 01:21:32.828559 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:21:32.828733 (+ 174us) server_negotiation.cc:407] Connection header received
0812 01:21:32.829193 (+ 460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:21:32.829194 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:21:32.829203 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:21:32.829254 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:21:32.829817 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:32.830291 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:32.831710 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:32.831908 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:32.831963 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:32.834132 (+ 2169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:21:32.834133 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:21:32.834134 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:21:32.834141 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:21:32.835655 (+ 1514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:21:32.836090 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:21:32.836092 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:21:32.836092 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:21:32.836156 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:21:32.836413 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:21:32.836414 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:21:32.836415 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:21:32.836528 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:21:32.836606 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 01:21:32.836877 (+ 271us) server_negotiation.cc:299] Negotiation successful
0812 01:21:32.836959 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 01:21:32.837255 14659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:32.828434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:36985 (local address 127.0.0.1:55458)
0812 01:21:32.828610 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:32.828651 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 01:21:32.829085 (+ 434us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:21:32.829308 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:21:32.829311 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:21:32.829549 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:21:32.829746 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:21:32.829750 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:21:32.830380 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:21:32.830381 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:21:32.831620 (+ 1239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:21:32.831624 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:21:32.831993 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:21:32.831994 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:21:32.832168 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:32.832667 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:21:32.832675 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:21:32.834034 (+ 1359us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:21:32.835760 (+ 1726us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:21:32.835768 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:21:32.835770 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:21:32.835996 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:21:32.836228 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:21:32.836229 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:21:32.836229 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:21:32.836342 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:21:32.836599 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:21:32.836606 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:21:32.836747 (+ 141us) client_negotiation.cc:769] Sending connection context
0812 01:21:32.836855 (+ 108us) client_negotiation.cc:240] Negotiation successful
0812 01:21:32.837006 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":98,"threads_started":1}
I20260812 01:21:32.837793 14656 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.0.189" port: 38889 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.24.0.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.0.188@KRBTEST.COM" created.
Aug 12 01:21:32 dist-test-slave-t3q3 krb5kdc[14386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497692, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.189@KRBTEST.COM for kudu/127.24.0.189@KRBTEST.COM
I20260812 01:21:32.845796 14659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:32.838070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.189:38889 (local address 127.0.0.1:39414)
0812 01:21:32.838136 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:32.838149 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 01:21:32.838196 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:21:32.838467 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:21:32.838469 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:21:32.838612 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:21:32.838675 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:21:32.838677 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:21:32.839506 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:21:32.839507 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:21:32.839985 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:21:32.839987 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:21:32.840552 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:21:32.840553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:21:32.840616 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:32.840926 (+ 310us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:21:32.840930 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:21:32.842611 (+ 1681us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:21:32.844738 (+ 2127us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:21:32.844738 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:21:32.844739 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:21:32.844902 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:21:32.845124 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:21:32.845125 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:21:32.845125 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:21:32.845178 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:21:32.845471 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:21:32.845471 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:21:32.845509 (+ 38us) client_negotiation.cc:769] Sending connection context
0812 01:21:32.845570 (+ 61us) client_negotiation.cc:240] Negotiation successful
0812 01:21:32.845666 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260812 01:21:32.845876 14661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:32.838119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39414 (local address 127.24.0.189:38889)
0812 01:21:32.838312 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0812 01:21:32.838317 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 01:21:32.838335 (+ 18us) server_negotiation.cc:407] Connection header received
0812 01:21:32.838363 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:21:32.838365 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:21:32.838375 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:21:32.838435 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:21:32.838754 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:32.839402 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:32.840088 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:32.840460 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:32.840528 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:32.842718 (+ 2190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:21:32.842726 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:21:32.842726 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:21:32.842732 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:21:32.844624 (+ 1892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:21:32.844994 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:21:32.844997 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:21:32.844997 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:21:32.845057 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:21:32.845256 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:21:32.845257 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:21:32.845257 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:21:32.845399 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:21:32.845491 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 01:21:32.845608 (+ 117us) server_negotiation.cc:299] Negotiation successful
0812 01:21:32.845728 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":44,"server-negotiator.queue_time_us":164,"thread_start_us":137,"threads_started":1}
I20260812 01:21:32.846395 14656 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.0.188" port: 33327 } has no permanent_uuid. Determining permanent_uuid...
I20260812 01:21:32.846771 14659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:32.846618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.188:33327 (local address 127.0.0.1:36828)
0812 01:21:32.846678 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:32.846736 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.188:33327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260812 01:21:32.846918 14568 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.0.188:33327: connect: Connection refused (error 111)
I20260812 01:21:32.847391 14659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:32.847293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.188:33327 (local address 127.0.0.1:36844)
0812 01:21:32.847347 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:32.847373 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.188:33327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260812 01:21:32.847519 14656 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.0.188:33327: Network error: Client connection negotiation failed: client connection to 127.24.0.188:33327: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.0.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.188.keytab.
Entry for principal kudu/127.24.0.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.188.keytab.
I20260812 01:21:32.856608 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786497534195564-24578-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786497534195564-24578-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786497534195564-24578-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786497534195564-24578-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.188:33327
--webserver_interface=127.24.0.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.188.keytab
--principal=kudu/127.24.0.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:35795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.0.190:36985,127.24.0.189:38889,127.24.0.188:33327
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 01:21:32.896768 14656 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.0.188" port: 33327 } attempt: 1
I20260812 01:21:32.897315 14659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:32.897173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.188:33327 (local address 127.0.0.1:36858)
0812 01:21:32.897242 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:32.897286 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.188:33327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260812 01:21:32.897682 14659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:32.897605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.188:33327 (local address 127.0.0.1:36862)
0812 01:21:32.897644 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:32.897667 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.188:33327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260812 01:21:32.897804 14656 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.0.188:33327: Network error: Client connection negotiation failed: client connection to 127.24.0.188:33327: connect: Connection refused (error 111)
I20260812 01:21:32.899044 14530 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.0.189" port: 38889 } attempt: 3
Aug 12 01:21:32 dist-test-slave-t3q3 krb5kdc[14386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497692, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.190@KRBTEST.COM for kudu/127.24.0.189@KRBTEST.COM
I20260812 01:21:32.906260 14661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:32.899397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39422 (local address 127.24.0.189:38889)
0812 01:21:32.899455 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0812 01:21:32.899457 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:21:32.899468 (+ 11us) server_negotiation.cc:407] Connection header received
0812 01:21:32.899546 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:21:32.899547 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:21:32.899559 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:21:32.899626 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:21:32.899993 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:32.900623 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:32.901342 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:32.901600 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:32.901660 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:32.903726 (+ 2066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:21:32.903727 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:21:32.903728 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:21:32.903735 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:21:32.905116 (+ 1381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:21:32.905485 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:21:32.905486 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:21:32.905486 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:21:32.905546 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:21:32.905747 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:21:32.905748 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:21:32.905749 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:21:32.905863 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:21:32.905935 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 01:21:32.906096 (+ 161us) server_negotiation.cc:299] Negotiation successful
0812 01:21:32.906171 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 01:21:32.906281 14534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:32.899337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.189:38889 (local address 127.0.0.1:39422)
0812 01:21:32.899389 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:32.899408 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 01:21:32.899479 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:21:32.899651 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:21:32.899652 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:21:32.899816 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:21:32.899880 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:21:32.899882 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:21:32.900706 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:21:32.900708 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:21:32.901256 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:21:32.901257 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:21:32.901672 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:21:32.901673 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:21:32.901755 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:32.902257 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:21:32.902261 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:21:32.903636 (+ 1375us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:21:32.905227 (+ 1591us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:21:32.905228 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:21:32.905228 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:21:32.905399 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:21:32.905615 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:21:32.905616 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:21:32.905616 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:21:32.905681 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:21:32.905939 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:21:32.905940 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:21:32.905986 (+ 46us) client_negotiation.cc:769] Sending connection context
0812 01:21:32.906071 (+ 85us) client_negotiation.cc:240] Negotiation successful
0812 01:21:32.906171 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260812 01:21:32.906852 14530 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.0.188" port: 33327 } has no permanent_uuid. Determining permanent_uuid...
I20260812 01:21:32.907228 14534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:32.907099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.188:33327 (local address 127.0.0.1:36864)
0812 01:21:32.907167 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:32.907204 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.188:33327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260812 01:21:32.907541 14534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:32.907481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.188:33327 (local address 127.0.0.1:36868)
0812 01:21:32.907503 (+ 22us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:32.907526 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.188:33327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
W20260812 01:21:32.907650 14530 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.0.188:33327: Network error: Client connection negotiation failed: client connection to 127.24.0.188:33327: connect: Connection refused (error 111)
I20260812 01:21:32.938789 14530 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.0.188" port: 33327 } attempt: 1
I20260812 01:21:32.939280 14534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:32.939129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.188:33327 (local address 127.0.0.1:36878)
0812 01:21:32.939218 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:32.939250 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.188:33327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260812 01:21:32.939708 14534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:32.939566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.188:33327 (local address 127.0.0.1:36894)
0812 01:21:32.939632 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:32.939676 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.188:33327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260812 01:21:32.939852 14530 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.0.188:33327: Network error: Client connection negotiation failed: client connection to 127.24.0.188:33327: connect: Connection refused (error 111)
W20260812 01:21:32.942310 14671 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:21:32.942492 14671 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:21:32.942541 14671 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:21:32.943658 14671 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:21:32.943698 14671 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:21:32.943713 14671 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:21:32.943749 14671 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:21:32.943775 14671 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:21:32.945324 14671 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:35795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786497534195564-24578-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786497534195564-24578-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.24.0.190:36985,127.24.0.189:38889,127.24.0.188:33327
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.188.keytab
--principal=kudu/127.24.0.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.188:33327
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786497534195564-24578-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.24.0.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14671
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786497534195564-24578-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:21:32.945575 14671 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:21:32.945822 14671 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:21:32.950647 14686 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:21:32.950647 14682 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:21:32.950668 14681 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:21:32.951136 14671 server_base.cc:1034] running on GCE node
I20260812 01:21:32.951375 14671 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:21:32.951630 14671 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:21:32.952824 14671 hybrid_clock.cc:648] HybridClock initialized: now 1786497692952810 us; error 27 us; skew 500 ppm
Aug 12 01:21:32 dist-test-slave-t3q3 krb5kdc[14386](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497692, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:21:32.955366 14671 init.cc:377] Logged in from keytab as kudu/127.24.0.188@KRBTEST.COM (short username kudu)
I20260812 01:21:32.955753 14671 webserver.cc:460] Webserver started at http://127.24.0.188:46675/ using document root <none> and password file <none>
I20260812 01:21:32.955947 14671 fs_manager.cc:359] Metadata directory not provided
I20260812 01:21:32.955986 14671 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:21:32.956140 14671 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:21:32.957100 14671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786497534195564-24578-0/minicluster-data/master-2/data/instance:
uuid: "f9b8bf20f70a4cd2a36e2a2150889483"
format_stamp: "Formatted at 2026-08-12 01:21:32 on dist-test-slave-t3q3"
server_key: "c17c9d43bdda3d9dd1d2c3a65481f1b3"
server_key_iv: "cd59d9ffb8f3570514f3dd0e26089e3d"
server_key_version: "encryptionkey@0"
I20260812 01:21:32.957391 14671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786497534195564-24578-0/minicluster-data/master-2/wal/instance:
uuid: "f9b8bf20f70a4cd2a36e2a2150889483"
format_stamp: "Formatted at 2026-08-12 01:21:32 on dist-test-slave-t3q3"
server_key: "c17c9d43bdda3d9dd1d2c3a65481f1b3"
server_key_iv: "cd59d9ffb8f3570514f3dd0e26089e3d"
server_key_version: "encryptionkey@0"
I20260812 01:21:32.958636 14671 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 01:21:32.959302 14698 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:21:32.959477 14671 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 01:21:32.959584 14671 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786497534195564-24578-0/minicluster-data/master-2/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786497534195564-24578-0/minicluster-data/master-2/wal
uuid: "f9b8bf20f70a4cd2a36e2a2150889483"
format_stamp: "Formatted at 2026-08-12 01:21:32 on dist-test-slave-t3q3"
server_key: "c17c9d43bdda3d9dd1d2c3a65481f1b3"
server_key_iv: "cd59d9ffb8f3570514f3dd0e26089e3d"
server_key_version: "encryptionkey@0"
I20260812 01:21:32.959672 14671 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786497534195564-24578-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786497534195564-24578-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786497534195564-24578-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:21:32.965939 14656 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.0.188" port: 33327 } attempt: 2
I20260812 01:21:32.966318 14659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:32.966191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.188:33327 (local address 127.0.0.1:36900)
0812 01:21:32.966258 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:32.966292 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.188:33327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260812 01:21:32.966638 14659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:32.966545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.188:33327 (local address 127.0.0.1:36914)
0812 01:21:32.966589 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:32.966615 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.188:33327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260812 01:21:32.966734 14656 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.0.188:33327: Network error: Client connection negotiation failed: client connection to 127.24.0.188:33327: connect: Connection refused (error 111)
I20260812 01:21:32.982771 14671 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:21:32.985329 14671 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:21:32.985498 14671 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:21:32.989750 14671 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.188:33327
I20260812 01:21:32.989789 14797 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.188:33327 every 8 connection(s)
I20260812 01:21:32.990149 14671 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786497534195564-24578-0/minicluster-data/master-2/data/info.pb
I20260812 01:21:32.991024 14799 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:21:32.991755 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 14671
I20260812 01:21:32.991879 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786497534195564-24578-0/minicluster-data/master-2/wal/instance
I20260812 01:21:32.992141 24578 external_mini_cluster.cc:1428] Setting key eb56b76997f017b7fbf8e98c7eabdb99
I20260812 01:21:32.993012 14799 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.0.190" port: 36985 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 01:21:32 dist-test-slave-t3q3 krb5kdc[14386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497692, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.188@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
Aug 12 01:21:32 dist-test-slave-t3q3 krb5kdc[14386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497692, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.0.188@KRBTEST.COM
I20260812 01:21:33.002817 14537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:32.994108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55466 (local address 127.24.0.190:36985)
0812 01:21:32.994188 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0812 01:21:32.994190 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:21:32.994358 (+ 168us) server_negotiation.cc:407] Connection header received
0812 01:21:32.994393 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:21:32.994394 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:21:32.994405 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:21:32.994469 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:21:32.994925 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:32.995635 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:32.996830 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:32.997120 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:32.997183 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:32.999517 (+ 2334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:21:32.999518 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:21:32.999519 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:21:32.999525 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:21:33.001321 (+ 1796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:21:33.001821 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:21:33.001822 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:21:33.001822 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:21:33.001872 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:21:33.002149 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:21:33.002150 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:21:33.002150 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:21:33.002264 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:21:33.002346 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 01:21:33.002635 (+ 289us) server_negotiation.cc:299] Negotiation successful
0812 01:21:33.002723 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 01:21:33.003057 14805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:32.994034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:36985 (local address 127.0.0.1:55466)
0812 01:21:32.994244 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:32.994289 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 01:21:32.994350 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:21:32.994494 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:21:32.994496 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:21:32.994742 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:21:32.994834 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:21:32.994838 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:21:32.995734 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:21:32.995735 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:21:32.996754 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:21:32.996756 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:21:32.997205 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:21:32.997206 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:21:32.997281 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:32.997761 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:21:32.997772 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:21:32.999431 (+ 1659us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:21:33.001400 (+ 1969us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:21:33.001407 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:21:33.001410 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:21:33.001744 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:21:33.001942 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:21:33.001943 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:21:33.001943 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:21:33.002080 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:21:33.002352 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:21:33.002360 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:21:33.002492 (+ 132us) client_negotiation.cc:769] Sending connection context
0812 01:21:33.002599 (+ 107us) client_negotiation.cc:240] Negotiation successful
0812 01:21:33.002764 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":115,"threads_started":1}
I20260812 01:21:33.003577 14803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:32.992702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36916 (local address 127.24.0.188:33327)
0812 01:21:32.992877 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0812 01:21:32.992885 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 01:21:32.992911 (+ 26us) server_negotiation.cc:407] Connection header received
0812 01:21:32.993325 (+ 414us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:21:32.993332 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:21:32.993378 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:21:32.993489 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:21:32.994278 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:32.995535 (+ 1257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:32.996598 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:32.997001 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:32.997120 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:33.000100 (+ 2980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:21:33.000107 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:21:33.000107 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:21:33.000114 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:21:33.002230 (+ 2116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:21:33.002568 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:21:33.002572 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:21:33.002572 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:21:33.002633 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:21:33.002855 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:21:33.002856 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:21:33.002856 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:21:33.003051 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:21:33.003194 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0812 01:21:33.003389 (+ 195us) server_negotiation.cc:299] Negotiation successful
0812 01:21:33.003469 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":89,"threads_started":1}
I20260812 01:21:33.003651 14799 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.0.189" port: 38889 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 01:21:33 dist-test-slave-t3q3 krb5kdc[14386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497692, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.188@KRBTEST.COM for kudu/127.24.0.189@KRBTEST.COM
Aug 12 01:21:33 dist-test-slave-t3q3 krb5kdc[14386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497692, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
I20260812 01:21:33.011296 14805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:33.004029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.189:38889 (local address 127.0.0.1:39438)
0812 01:21:33.004096 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:33.004109 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 01:21:33.004218 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:21:33.004352 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:21:33.004353 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:21:33.004509 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:21:33.004551 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:21:33.004553 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:21:33.005259 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:21:33.005260 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:21:33.005832 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:21:33.005834 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:21:33.006167 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:21:33.006168 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:21:33.006224 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:33.006653 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:21:33.006657 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:21:33.007938 (+ 1281us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:21:33.010255 (+ 2317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:21:33.010256 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:21:33.010256 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:21:33.010491 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:21:33.010692 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:21:33.010693 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:21:33.010693 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:21:33.010758 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:21:33.010995 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:21:33.010996 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:21:33.011046 (+ 50us) client_negotiation.cc:769] Sending connection context
0812 01:21:33.011130 (+ 84us) client_negotiation.cc:240] Negotiation successful
0812 01:21:33.011230 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260812 01:21:33.011379 14661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:33.004116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39438 (local address 127.24.0.189:38889)
0812 01:21:33.004179 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0812 01:21:33.004181 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:21:33.004191 (+ 10us) server_negotiation.cc:407] Connection header received
0812 01:21:33.004283 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:21:33.004284 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:21:33.004297 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:21:33.004348 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:21:33.004631 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:33.005178 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:33.005914 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:33.006099 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:33.006156 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:33.008037 (+ 1881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:21:33.008038 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:21:33.008038 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:21:33.008045 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:21:33.010147 (+ 2102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:21:33.010570 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:21:33.010571 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:21:33.010571 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:21:33.010623 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:21:33.010826 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:21:33.010827 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:21:33.010827 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:21:33.010929 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:21:33.010979 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 01:21:33.011170 (+ 191us) server_negotiation.cc:299] Negotiation successful
0812 01:21:33.011267 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260812 01:21:33.011714 14799 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.0.188" port: 33327 } has no permanent_uuid. Determining permanent_uuid...
I20260812 01:21:33.012365 14537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:33.004501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55472 (local address 127.24.0.190:36985)
0812 01:21:33.004534 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0812 01:21:33.004536 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:21:33.004543 (+ 7us) server_negotiation.cc:407] Connection header received
0812 01:21:33.004561 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:21:33.004561 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:21:33.004569 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:21:33.004628 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:21:33.005047 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:33.005559 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:33.006390 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:33.006675 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:33.006741 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:33.009064 (+ 2323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:21:33.009065 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:21:33.009065 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:21:33.009071 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:21:33.011087 (+ 2016us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:21:33.011478 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:21:33.011479 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:21:33.011479 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:21:33.011537 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:21:33.011748 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:21:33.011749 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:21:33.011749 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:21:33.011886 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:21:33.012103 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0812 01:21:33.012145 (+ 42us) server_negotiation.cc:299] Negotiation successful
0812 01:21:33.012258 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260812 01:21:33.015017 14530 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.0.188" port: 33327 } attempt: 2
Aug 12 01:21:33 dist-test-slave-t3q3 krb5kdc[14386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497692, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.188@KRBTEST.COM for kudu/127.24.0.188@KRBTEST.COM
Aug 12 01:21:33 dist-test-slave-t3q3 krb5kdc[14386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497692, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.0.189@KRBTEST.COM
I20260812 01:21:33.019361 14805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:33.011972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.188:33327 (local address 127.0.0.1:36922)
0812 01:21:33.012139 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:33.012148 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 01:21:33.012195 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:21:33.012393 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:21:33.012394 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:21:33.012549 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:21:33.012585 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:21:33.012587 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:21:33.013382 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:21:33.013383 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:21:33.014009 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:21:33.014012 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:21:33.014480 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:21:33.014481 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:21:33.014533 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:33.014869 (+ 336us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:21:33.014874 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:21:33.016402 (+ 1528us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:21:33.018081 (+ 1679us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:21:33.018082 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:21:33.018082 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:21:33.018305 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:21:33.018692 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:21:33.018692 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:21:33.018693 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:21:33.018754 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:21:33.019007 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:21:33.019008 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:21:33.019071 (+ 63us) client_negotiation.cc:769] Sending connection context
0812 01:21:33.019152 (+ 81us) client_negotiation.cc:240] Negotiation successful
0812 01:21:33.019252 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149}
I20260812 01:21:33.019423 14803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:33.012039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36922 (local address 127.24.0.188:33327)
0812 01:21:33.012110 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 01:21:33.012112 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:21:33.012200 (+ 88us) server_negotiation.cc:407] Connection header received
0812 01:21:33.012248 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:21:33.012249 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:21:33.012260 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:21:33.012317 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:21:33.012651 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:33.013309 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:33.014085 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:33.014411 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:33.014461 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:33.016479 (+ 2018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:21:33.016481 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:21:33.016481 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:21:33.016488 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:21:33.017986 (+ 1498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:21:33.018370 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:21:33.018371 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:21:33.018372 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:21:33.018423 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:21:33.018827 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:21:33.018828 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:21:33.018829 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:21:33.018939 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:21:33.019014 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 01:21:33.019185 (+ 171us) server_negotiation.cc:299] Negotiation successful
0812 01:21:33.019310 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":46,"server-negotiator.queue_time_us":53}
Aug 12 01:21:33 dist-test-slave-t3q3 krb5kdc[14386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497692, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.190@KRBTEST.COM for kudu/127.24.0.188@KRBTEST.COM
I20260812 01:21:33.021754 14799 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9b8bf20f70a4cd2a36e2a2150889483: Bootstrap starting.
I20260812 01:21:33.022431 14661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:33.013019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39444 (local address 127.24.0.189:38889)
0812 01:21:33.013060 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 01:21:33.013062 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:21:33.013071 (+ 9us) server_negotiation.cc:407] Connection header received
0812 01:21:33.013102 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:21:33.013103 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:21:33.013111 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:21:33.013156 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:21:33.013743 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:33.014258 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:33.015134 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:33.015423 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:33.015491 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:33.019504 (+ 4013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:21:33.019506 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:21:33.019506 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:21:33.019513 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:21:33.021209 (+ 1696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:21:33.021584 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:21:33.021585 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:21:33.021585 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:21:33.021634 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:21:33.021890 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:21:33.021892 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:21:33.021892 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:21:33.022002 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:21:33.022192 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0812 01:21:33.022242 (+ 50us) server_negotiation.cc:299] Negotiation successful
0812 01:21:33.022319 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 01:21:33.022531 14799 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f9b8bf20f70a4cd2a36e2a2150889483: Neither blocks nor log segments found. Creating new log.
I20260812 01:21:33.022881 14799 log.cc:824] T 00000000000000000000000000000000 P f9b8bf20f70a4cd2a36e2a2150889483: Log is configured to *not* fsync() on all Append() calls
I20260812 01:21:33.023494 14534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:33.015300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.188:33327 (local address 127.0.0.1:36932)
0812 01:21:33.015350 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:33.015360 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 01:21:33.015397 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:21:33.015664 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:21:33.015665 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:21:33.015827 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:21:33.015861 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:21:33.015863 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:21:33.017333 (+ 1470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:21:33.017334 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:21:33.017813 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:21:33.017814 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:21:33.018298 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:21:33.018300 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:21:33.018367 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:33.018696 (+ 329us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:21:33.018701 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:21:33.020036 (+ 1335us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:21:33.022339 (+ 2303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:21:33.022339 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:21:33.022340 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:21:33.022567 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:21:33.022811 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:21:33.022811 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:21:33.022812 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:21:33.022871 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:21:33.023201 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:21:33.023203 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:21:33.023248 (+ 45us) client_negotiation.cc:769] Sending connection context
0812 01:21:33.023295 (+ 47us) client_negotiation.cc:240] Negotiation successful
0812 01:21:33.023396 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260812 01:21:33.023833 24578 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 01:21:33.023840 14799 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9b8bf20f70a4cd2a36e2a2150889483: No bootstrap required, opened a new log
I20260812 01:21:33.024080 14806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:33.015350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36932 (local address 127.24.0.188:33327)
0812 01:21:33.015529 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0812 01:21:33.015531 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:21:33.015538 (+ 7us) server_negotiation.cc:407] Connection header received
0812 01:21:33.015558 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:21:33.015559 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:21:33.015568 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:21:33.015635 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:21:33.015957 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:33.017112 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:33.017905 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:33.018210 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:33.018282 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:33.020255 (+ 1973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:21:33.020256 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:21:33.020257 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:21:33.020263 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:21:33.022238 (+ 1975us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:21:33.022666 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:21:33.022667 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:21:33.022667 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:21:33.022732 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:21:33.022949 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:21:33.022949 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:21:33.022950 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:21:33.023098 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:21:33.023817 (+ 719us) server_negotiation.cc:1035] Waiting for connection context
0812 01:21:33.023861 (+ 44us) server_negotiation.cc:299] Negotiation successful
0812 01:21:33.023941 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":107,"threads_started":1}
I20260812 01:21:33.025439 14530 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac1fb52d504340dfa69635fd8a594ef0: Bootstrap starting.
I20260812 01:21:33.026156 14530 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ac1fb52d504340dfa69635fd8a594ef0: Neither blocks nor log segments found. Creating new log.
I20260812 01:21:33.026054 14799 raft_consensus.cc:348] T 00000000000000000000000000000000 P f9b8bf20f70a4cd2a36e2a2150889483 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac1fb52d504340dfa69635fd8a594ef0" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 36985 } } peers { permanent_uuid: "0435323d5c1c411aa9c9163a986d5d0f" member_type: VOTER last_known_addr { host: "127.24.0.189" port: 38889 } } peers { permanent_uuid: "f9b8bf20f70a4cd2a36e2a2150889483" member_type: VOTER last_known_addr { host: "127.24.0.188" port: 33327 } }
I20260812 01:21:33.026206 14799 raft_consensus.cc:374] T 00000000000000000000000000000000 P f9b8bf20f70a4cd2a36e2a2150889483 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:21:33.026244 14799 raft_consensus.cc:729] T 00000000000000000000000000000000 P f9b8bf20f70a4cd2a36e2a2150889483 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9b8bf20f70a4cd2a36e2a2150889483, State: Initialized, Role: FOLLOWER
I20260812 01:21:33.026397 14799 consensus_queue.cc:260] T 00000000000000000000000000000000 P f9b8bf20f70a4cd2a36e2a2150889483 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac1fb52d504340dfa69635fd8a594ef0" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 36985 } } peers { permanent_uuid: "0435323d5c1c411aa9c9163a986d5d0f" member_type: VOTER last_known_addr { host: "127.24.0.189" port: 38889 } } peers { permanent_uuid: "f9b8bf20f70a4cd2a36e2a2150889483" member_type: VOTER last_known_addr { host: "127.24.0.188" port: 33327 } }
I20260812 01:21:33.026443 14530 log.cc:824] T 00000000000000000000000000000000 P ac1fb52d504340dfa69635fd8a594ef0: Log is configured to *not* fsync() on all Append() calls
I20260812 01:21:33.026723 14799 sys_catalog.cc:564] T 00000000000000000000000000000000 P f9b8bf20f70a4cd2a36e2a2150889483 [sys.catalog]: configured and running, proceeding with master startup.
W20260812 01:21:33.029325 14842 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f9b8bf20f70a4cd2a36e2a2150889483: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 01:21:33.029429 14842 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 01:21:33.028684 14671 master_runner.cc:186] Error getting master registration for 127.24.0.190:36985: OK, instance_id { permanent_uuid: "ac1fb52d504340dfa69635fd8a594ef0" instance_seqno: 1786497692666436 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 01:21:33.030951 14671 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 01:21:33.031636 14530 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac1fb52d504340dfa69635fd8a594ef0: No bootstrap required, opened a new log
I20260812 01:21:33.034452 14530 raft_consensus.cc:348] T 00000000000000000000000000000000 P ac1fb52d504340dfa69635fd8a594ef0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac1fb52d504340dfa69635fd8a594ef0" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 36985 } } peers { permanent_uuid: "0435323d5c1c411aa9c9163a986d5d0f" member_type: VOTER last_known_addr { host: "127.24.0.189" port: 38889 } } peers { permanent_uuid: "f9b8bf20f70a4cd2a36e2a2150889483" member_type: VOTER last_known_addr { host: "127.24.0.188" port: 33327 } }
I20260812 01:21:33.034855 14537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:33.025567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55480 (local address 127.24.0.190:36985)
0812 01:21:33.025647 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0812 01:21:33.025649 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:21:33.025656 (+ 7us) server_negotiation.cc:407] Connection header received
0812 01:21:33.025674 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:21:33.025675 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:21:33.025683 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:21:33.025745 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:21:33.026507 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:33.027273 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:33.028350 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:33.028624 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:33.028663 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:33.030544 (+ 1881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:21:33.030545 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:21:33.030546 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:21:33.030553 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:21:33.032974 (+ 2421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:21:33.033394 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:21:33.033395 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:21:33.033395 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:21:33.033722 (+ 327us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:21:33.034219 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:21:33.034220 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:21:33.034220 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:21:33.034354 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:21:33.034426 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 01:21:33.034654 (+ 228us) server_negotiation.cc:299] Negotiation successful
0812 01:21:33.034746 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 01:21:33.035557 14530 raft_consensus.cc:374] T 00000000000000000000000000000000 P ac1fb52d504340dfa69635fd8a594ef0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:21:33.035912 14530 raft_consensus.cc:729] T 00000000000000000000000000000000 P ac1fb52d504340dfa69635fd8a594ef0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac1fb52d504340dfa69635fd8a594ef0, State: Initialized, Role: FOLLOWER
I20260812 01:21:33.036108 14530 consensus_queue.cc:260] T 00000000000000000000000000000000 P ac1fb52d504340dfa69635fd8a594ef0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac1fb52d504340dfa69635fd8a594ef0" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 36985 } } peers { permanent_uuid: "0435323d5c1c411aa9c9163a986d5d0f" member_type: VOTER last_known_addr { host: "127.24.0.189" port: 38889 } } peers { permanent_uuid: "f9b8bf20f70a4cd2a36e2a2150889483" member_type: VOTER last_known_addr { host: "127.24.0.188" port: 33327 } }
I20260812 01:21:33.036429 14844 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac1fb52d504340dfa69635fd8a594ef0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac1fb52d504340dfa69635fd8a594ef0" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 36985 } } peers { permanent_uuid: "0435323d5c1c411aa9c9163a986d5d0f" member_type: VOTER last_known_addr { host: "127.24.0.189" port: 38889 } } peers { permanent_uuid: "f9b8bf20f70a4cd2a36e2a2150889483" member_type: VOTER last_known_addr { host: "127.24.0.188" port: 33327 } } }
I20260812 01:21:33.036589 14844 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac1fb52d504340dfa69635fd8a594ef0 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 01:21:33.036711 14806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:33.025603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36944 (local address 127.24.0.188:33327)
0812 01:21:33.025639 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0812 01:21:33.025640 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 01:21:33.026011 (+ 371us) server_negotiation.cc:407] Connection header received
0812 01:21:33.026031 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:21:33.026032 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:21:33.026056 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:21:33.026103 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:21:33.026920 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:33.027688 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:33.031461 (+ 3773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:33.031735 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:33.031798 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:33.033213 (+ 1415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:21:33.033214 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:21:33.033215 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:21:33.033221 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:21:33.034878 (+ 1657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:21:33.035287 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:21:33.035288 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:21:33.035289 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:21:33.035349 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:21:33.035585 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:21:33.035586 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:21:33.035586 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:21:33.035708 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:21:33.036193 (+ 485us) server_negotiation.cc:1035] Waiting for connection context
0812 01:21:33.036504 (+ 311us) server_negotiation.cc:299] Negotiation successful
0812 01:21:33.036591 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260812 01:21:33.036492 14530 sys_catalog.cc:564] T 00000000000000000000000000000000 P ac1fb52d504340dfa69635fd8a594ef0 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:21:33.037743 14661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:33.025228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39458 (local address 127.24.0.189:38889)
0812 01:21:33.025265 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0812 01:21:33.025268 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:21:33.025863 (+ 595us) server_negotiation.cc:407] Connection header received
0812 01:21:33.026234 (+ 371us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:21:33.026235 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:21:33.026246 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:21:33.026276 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:21:33.027569 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:33.028284 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:33.029178 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:33.029482 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:33.029531 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:33.030986 (+ 1455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:21:33.030987 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:21:33.030987 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:21:33.030994 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:21:33.035358 (+ 4364us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:21:33.036552 (+ 1194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:21:33.036554 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:21:33.036554 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:21:33.036610 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:21:33.036948 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:21:33.036949 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:21:33.036949 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:21:33.037047 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:21:33.037164 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0812 01:21:33.037530 (+ 366us) server_negotiation.cc:299] Negotiation successful
0812 01:21:33.037606 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
W20260812 01:21:33.039598 14860 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ac1fb52d504340dfa69635fd8a594ef0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 01:21:33.039764 14860 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 01:21:33.040139 14822 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9b8bf20f70a4cd2a36e2a2150889483 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac1fb52d504340dfa69635fd8a594ef0" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 36985 } } peers { permanent_uuid: "0435323d5c1c411aa9c9163a986d5d0f" member_type: VOTER last_known_addr { host: "127.24.0.189" port: 38889 } } peers { permanent_uuid: "f9b8bf20f70a4cd2a36e2a2150889483" member_type: VOTER last_known_addr { host: "127.24.0.188" port: 33327 } } }
I20260812 01:21:33.040233 14822 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9b8bf20f70a4cd2a36e2a2150889483 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 01:21:33.040161 14409 master_runner.cc:186] Error getting master registration for 127.24.0.189:38889: OK, instance_id { permanent_uuid: "0435323d5c1c411aa9c9163a986d5d0f" instance_seqno: 1786497692823649 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 01:21:33.042488 14409 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 01:21:33.057896 14656 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.0.188" port: 33327 } attempt: 3
Aug 12 01:21:33 dist-test-slave-t3q3 krb5kdc[14386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497692, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.189@KRBTEST.COM for kudu/127.24.0.188@KRBTEST.COM
I20260812 01:21:33.066092 14659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:33.058167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.188:33327 (local address 127.0.0.1:36956)
0812 01:21:33.058223 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:33.058234 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 01:21:33.058269 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:21:33.058428 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:21:33.058429 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:21:33.058616 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:21:33.058665 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:21:33.058666 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:21:33.059610 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:21:33.059612 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:21:33.060440 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:21:33.060443 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:21:33.060919 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:21:33.060920 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:21:33.060980 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:33.061434 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:21:33.061439 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:21:33.063049 (+ 1610us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:21:33.064970 (+ 1921us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:21:33.064971 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:21:33.064972 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:21:33.065208 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:21:33.065425 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:21:33.065426 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:21:33.065427 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:21:33.065495 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:21:33.065767 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:21:33.065768 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:21:33.065816 (+ 48us) client_negotiation.cc:769] Sending connection context
0812 01:21:33.065892 (+ 76us) client_negotiation.cc:240] Negotiation successful
0812 01:21:33.065990 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260812 01:21:33.066102 14806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:33.058232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36956 (local address 127.24.0.188:33327)
0812 01:21:33.058273 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 01:21:33.058275 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:21:33.058311 (+ 36us) server_negotiation.cc:407] Connection header received
0812 01:21:33.058333 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:21:33.058334 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:21:33.058345 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:21:33.058399 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:21:33.058748 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:33.059522 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:33.060528 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:33.060833 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:33.060907 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:33.063138 (+ 2231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:21:33.063139 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:21:33.063140 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:21:33.063147 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:21:33.064881 (+ 1734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:21:33.065288 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:21:33.065289 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:21:33.065289 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:21:33.065349 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:21:33.065565 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:21:33.065566 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:21:33.065566 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:21:33.065695 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:21:33.065763 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 01:21:33.065930 (+ 167us) server_negotiation.cc:299] Negotiation successful
0812 01:21:33.066007 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 01:21:33.067317 14844 raft_consensus.cc:482] T 00000000000000000000000000000000 P ac1fb52d504340dfa69635fd8a594ef0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 01:21:33.067415 14844 raft_consensus.cc:504] T 00000000000000000000000000000000 P ac1fb52d504340dfa69635fd8a594ef0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac1fb52d504340dfa69635fd8a594ef0" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 36985 } } peers { permanent_uuid: "0435323d5c1c411aa9c9163a986d5d0f" member_type: VOTER last_known_addr { host: "127.24.0.189" port: 38889 } } peers { permanent_uuid: "f9b8bf20f70a4cd2a36e2a2150889483" member_type: VOTER last_known_addr { host: "127.24.0.188" port: 33327 } }
I20260812 01:21:33.067693 14844 leader_election.cc:288] T 00000000000000000000000000000000 P ac1fb52d504340dfa69635fd8a594ef0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0435323d5c1c411aa9c9163a986d5d0f (127.24.0.189:38889), f9b8bf20f70a4cd2a36e2a2150889483 (127.24.0.188:33327)
I20260812 01:21:33.068022 14746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ac1fb52d504340dfa69635fd8a594ef0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9b8bf20f70a4cd2a36e2a2150889483" is_pre_election: true
I20260812 01:21:33.068176 14746 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f9b8bf20f70a4cd2a36e2a2150889483 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ac1fb52d504340dfa69635fd8a594ef0 in term 0.
I20260812 01:21:33.068392 14436 leader_election.cc:302] T 00000000000000000000000000000000 P ac1fb52d504340dfa69635fd8a594ef0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ac1fb52d504340dfa69635fd8a594ef0, f9b8bf20f70a4cd2a36e2a2150889483; no voters:
I20260812 01:21:33.068518 14656 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0435323d5c1c411aa9c9163a986d5d0f: Bootstrap starting.
I20260812 01:21:33.068545 14844 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ac1fb52d504340dfa69635fd8a594ef0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 01:21:33.068586 14844 raft_consensus.cc:482] T 00000000000000000000000000000000 P ac1fb52d504340dfa69635fd8a594ef0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 01:21:33.068608 14844 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ac1fb52d504340dfa69635fd8a594ef0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:21:33.069283 14656 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0435323d5c1c411aa9c9163a986d5d0f: Neither blocks nor log segments found. Creating new log.
I20260812 01:21:33.069478 14844 raft_consensus.cc:504] T 00000000000000000000000000000000 P ac1fb52d504340dfa69635fd8a594ef0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac1fb52d504340dfa69635fd8a594ef0" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 36985 } } peers { permanent_uuid: "0435323d5c1c411aa9c9163a986d5d0f" member_type: VOTER last_known_addr { host: "127.24.0.189" port: 38889 } } peers { permanent_uuid: "f9b8bf20f70a4cd2a36e2a2150889483" member_type: VOTER last_known_addr { host: "127.24.0.188" port: 33327 } }
I20260812 01:21:33.069572 14656 log.cc:824] T 00000000000000000000000000000000 P 0435323d5c1c411aa9c9163a986d5d0f: Log is configured to *not* fsync() on all Append() calls
I20260812 01:21:33.069592 14844 leader_election.cc:288] T 00000000000000000000000000000000 P ac1fb52d504340dfa69635fd8a594ef0 [CANDIDATE]: Term 1 election: Requested vote from peers 0435323d5c1c411aa9c9163a986d5d0f (127.24.0.189:38889), f9b8bf20f70a4cd2a36e2a2150889483 (127.24.0.188:33327)
I20260812 01:21:33.069756 14746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ac1fb52d504340dfa69635fd8a594ef0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9b8bf20f70a4cd2a36e2a2150889483"
I20260812 01:21:33.069815 14746 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f9b8bf20f70a4cd2a36e2a2150889483 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:21:33.070636 14746 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f9b8bf20f70a4cd2a36e2a2150889483 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac1fb52d504340dfa69635fd8a594ef0 in term 1.
I20260812 01:21:33.070808 14436 leader_election.cc:302] T 00000000000000000000000000000000 P ac1fb52d504340dfa69635fd8a594ef0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ac1fb52d504340dfa69635fd8a594ef0, f9b8bf20f70a4cd2a36e2a2150889483; no voters:
I20260812 01:21:33.071017 14844 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ac1fb52d504340dfa69635fd8a594ef0 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:21:33.071163 14844 raft_consensus.cc:686] T 00000000000000000000000000000000 P ac1fb52d504340dfa69635fd8a594ef0 [term 1 LEADER]: Becoming Leader. State: Replica: ac1fb52d504340dfa69635fd8a594ef0, State: Running, Role: LEADER
I20260812 01:21:33.071257 14844 consensus_queue.cc:237] T 00000000000000000000000000000000 P ac1fb52d504340dfa69635fd8a594ef0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac1fb52d504340dfa69635fd8a594ef0" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 36985 } } peers { permanent_uuid: "0435323d5c1c411aa9c9163a986d5d0f" member_type: VOTER last_known_addr { host: "127.24.0.189" port: 38889 } } peers { permanent_uuid: "f9b8bf20f70a4cd2a36e2a2150889483" member_type: VOTER last_known_addr { host: "127.24.0.188" port: 33327 } }
I20260812 01:21:33.071640 14844 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac1fb52d504340dfa69635fd8a594ef0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ac1fb52d504340dfa69635fd8a594ef0. Latest consensus state: current_term: 1 leader_uuid: "ac1fb52d504340dfa69635fd8a594ef0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac1fb52d504340dfa69635fd8a594ef0" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 36985 } } peers { permanent_uuid: "0435323d5c1c411aa9c9163a986d5d0f" member_type: VOTER last_known_addr { host: "127.24.0.189" port: 38889 } } peers { permanent_uuid: "f9b8bf20f70a4cd2a36e2a2150889483" member_type: VOTER last_known_addr { host: "127.24.0.188" port: 33327 } } }
I20260812 01:21:33.071707 14844 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac1fb52d504340dfa69635fd8a594ef0 [sys.catalog]: This master's current role is: LEADER
I20260812 01:21:33.072566 14656 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0435323d5c1c411aa9c9163a986d5d0f: No bootstrap required, opened a new log
I20260812 01:21:33.074503 14656 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0435323d5c1c411aa9c9163a986d5d0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac1fb52d504340dfa69635fd8a594ef0" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 36985 } } peers { permanent_uuid: "0435323d5c1c411aa9c9163a986d5d0f" member_type: VOTER last_known_addr { host: "127.24.0.189" port: 38889 } } peers { permanent_uuid: "f9b8bf20f70a4cd2a36e2a2150889483" member_type: VOTER last_known_addr { host: "127.24.0.188" port: 33327 } }
I20260812 01:21:33.074626 14656 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0435323d5c1c411aa9c9163a986d5d0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:21:33.074659 14656 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0435323d5c1c411aa9c9163a986d5d0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0435323d5c1c411aa9c9163a986d5d0f, State: Initialized, Role: FOLLOWER
I20260812 01:21:33.074769 14656 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0435323d5c1c411aa9c9163a986d5d0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac1fb52d504340dfa69635fd8a594ef0" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 36985 } } peers { permanent_uuid: "0435323d5c1c411aa9c9163a986d5d0f" member_type: VOTER last_known_addr { host: "127.24.0.189" port: 38889 } } peers { permanent_uuid: "f9b8bf20f70a4cd2a36e2a2150889483" member_type: VOTER last_known_addr { host: "127.24.0.188" port: 33327 } }
I20260812 01:21:33.075048 14656 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0435323d5c1c411aa9c9163a986d5d0f [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:21:33.075111 14606 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ac1fb52d504340dfa69635fd8a594ef0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0435323d5c1c411aa9c9163a986d5d0f" is_pre_election: true
I20260812 01:21:33.075250 14606 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0435323d5c1c411aa9c9163a986d5d0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ac1fb52d504340dfa69635fd8a594ef0 in term 0.
I20260812 01:21:33.075059 14874 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0435323d5c1c411aa9c9163a986d5d0f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac1fb52d504340dfa69635fd8a594ef0" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 36985 } } peers { permanent_uuid: "0435323d5c1c411aa9c9163a986d5d0f" member_type: VOTER last_known_addr { host: "127.24.0.189" port: 38889 } } peers { permanent_uuid: "f9b8bf20f70a4cd2a36e2a2150889483" member_type: VOTER last_known_addr { host: "127.24.0.188" port: 33327 } } }
I20260812 01:21:33.075373 14874 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0435323d5c1c411aa9c9163a986d5d0f [sys.catalog]: This master's current role is: FOLLOWER
I20260812 01:21:33.076320 14604 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ac1fb52d504340dfa69635fd8a594ef0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0435323d5c1c411aa9c9163a986d5d0f"
I20260812 01:21:33.076401 14604 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0435323d5c1c411aa9c9163a986d5d0f [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:21:33.077174 14604 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0435323d5c1c411aa9c9163a986d5d0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac1fb52d504340dfa69635fd8a594ef0 in term 1.
W20260812 01:21:33.078418 14896 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0435323d5c1c411aa9c9163a986d5d0f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260812 01:21:33.078423 14873 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
W20260812 01:21:33.078480 14896 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 01:21:33.078686 14873 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:21:33.080399 14604 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0435323d5c1c411aa9c9163a986d5d0f [term 1 FOLLOWER]: Refusing update from remote peer ac1fb52d504340dfa69635fd8a594ef0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:21:33.080734 14844 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ac1fb52d504340dfa69635fd8a594ef0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0435323d5c1c411aa9c9163a986d5d0f" member_type: VOTER last_known_addr { host: "127.24.0.189" port: 38889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:21:33.081149 14746 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f9b8bf20f70a4cd2a36e2a2150889483 [term 1 FOLLOWER]: Refusing update from remote peer ac1fb52d504340dfa69635fd8a594ef0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:21:33.082304 14874 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0435323d5c1c411aa9c9163a986d5d0f [sys.catalog]: SysCatalogTable state changed. Reason: New leader ac1fb52d504340dfa69635fd8a594ef0. Latest consensus state: current_term: 1 leader_uuid: "ac1fb52d504340dfa69635fd8a594ef0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac1fb52d504340dfa69635fd8a594ef0" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 36985 } } peers { permanent_uuid: "0435323d5c1c411aa9c9163a986d5d0f" member_type: VOTER last_known_addr { host: "127.24.0.189" port: 38889 } } peers { permanent_uuid: "f9b8bf20f70a4cd2a36e2a2150889483" member_type: VOTER last_known_addr { host: "127.24.0.188" port: 33327 } } }
I20260812 01:21:33.082381 14874 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0435323d5c1c411aa9c9163a986d5d0f [sys.catalog]: This master's current role is: FOLLOWER
I20260812 01:21:33.082486 14844 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ac1fb52d504340dfa69635fd8a594ef0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9b8bf20f70a4cd2a36e2a2150889483" member_type: VOTER last_known_addr { host: "127.24.0.188" port: 33327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:21:33.083673 14822 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9b8bf20f70a4cd2a36e2a2150889483 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ac1fb52d504340dfa69635fd8a594ef0. Latest consensus state: current_term: 1 leader_uuid: "ac1fb52d504340dfa69635fd8a594ef0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac1fb52d504340dfa69635fd8a594ef0" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 36985 } } peers { permanent_uuid: "0435323d5c1c411aa9c9163a986d5d0f" member_type: VOTER last_known_addr { host: "127.24.0.189" port: 38889 } } peers { permanent_uuid: "f9b8bf20f70a4cd2a36e2a2150889483" member_type: VOTER last_known_addr { host: "127.24.0.188" port: 33327 } } }
I20260812 01:21:33.083771 14822 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9b8bf20f70a4cd2a36e2a2150889483 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 01:21:33.084707 14870 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac1fb52d504340dfa69635fd8a594ef0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ac1fb52d504340dfa69635fd8a594ef0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac1fb52d504340dfa69635fd8a594ef0" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 36985 } } peers { permanent_uuid: "0435323d5c1c411aa9c9163a986d5d0f" member_type: VOTER last_known_addr { host: "127.24.0.189" port: 38889 } } peers { permanent_uuid: "f9b8bf20f70a4cd2a36e2a2150889483" member_type: VOTER last_known_addr { host: "127.24.0.188" port: 33327 } } }
I20260812 01:21:33.084857 14870 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac1fb52d504340dfa69635fd8a594ef0 [sys.catalog]: This master's current role is: LEADER
I20260812 01:21:33.084964 14898 mvcc.cc:205] Tried to move back new op lower bound from 7317494550854332416 to 7317494550820458496. Current Snapshot: MvccSnapshot[applied={T|T < 7317494550854332416}]
I20260812 01:21:33.085022 14874 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0435323d5c1c411aa9c9163a986d5d0f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ac1fb52d504340dfa69635fd8a594ef0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac1fb52d504340dfa69635fd8a594ef0" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 36985 } } peers { permanent_uuid: "0435323d5c1c411aa9c9163a986d5d0f" member_type: VOTER last_known_addr { host: "127.24.0.189" port: 38889 } } peers { permanent_uuid: "f9b8bf20f70a4cd2a36e2a2150889483" member_type: VOTER last_known_addr { host: "127.24.0.188" port: 33327 } } }
I20260812 01:21:33.085103 14874 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0435323d5c1c411aa9c9163a986d5d0f [sys.catalog]: This master's current role is: FOLLOWER
I20260812 01:21:33.085278 14900 mvcc.cc:205] Tried to move back new op lower bound from 7317494550854332416 to 7317494550820458496. Current Snapshot: MvccSnapshot[applied={T|T < 7317494550854332416}]
I20260812 01:21:33.085640 14873 catalog_manager.cc:1347] Generated new cluster ID: 042352d4e68844d3a36d4c6868bf30cf
I20260812 01:21:33.085685 14873 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 01:21:33.092265 14844 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac1fb52d504340dfa69635fd8a594ef0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ac1fb52d504340dfa69635fd8a594ef0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac1fb52d504340dfa69635fd8a594ef0" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 36985 } } peers { permanent_uuid: "0435323d5c1c411aa9c9163a986d5d0f" member_type: VOTER last_known_addr { host: "127.24.0.189" port: 38889 } } peers { permanent_uuid: "f9b8bf20f70a4cd2a36e2a2150889483" member_type: VOTER last_known_addr { host: "127.24.0.188" port: 33327 } } }
I20260812 01:21:33.092344 14844 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac1fb52d504340dfa69635fd8a594ef0 [sys.catalog]: This master's current role is: LEADER
I20260812 01:21:33.092759 14822 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9b8bf20f70a4cd2a36e2a2150889483 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ac1fb52d504340dfa69635fd8a594ef0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac1fb52d504340dfa69635fd8a594ef0" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 36985 } } peers { permanent_uuid: "0435323d5c1c411aa9c9163a986d5d0f" member_type: VOTER last_known_addr { host: "127.24.0.189" port: 38889 } } peers { permanent_uuid: "f9b8bf20f70a4cd2a36e2a2150889483" member_type: VOTER last_known_addr { host: "127.24.0.188" port: 33327 } } }
I20260812 01:21:33.092837 14822 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9b8bf20f70a4cd2a36e2a2150889483 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 01:21:33.093238 14904 mvcc.cc:205] Tried to move back new op lower bound from 7317494550854332416 to 7317494550820458496. Current Snapshot: MvccSnapshot[applied={T|T < 7317494550854332416}]
I20260812 01:21:33.097033 14873 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:21:33.097790 14873 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:21:33.106293 14873 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ac1fb52d504340dfa69635fd8a594ef0: Generated new TSK 0
I20260812 01:21:33.106524 14873 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 01:21:34.029958 14842 catalog_manager.cc:1259] Loaded cluster ID: 042352d4e68844d3a36d4c6868bf30cf
I20260812 01:21:34.030058 14842 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f9b8bf20f70a4cd2a36e2a2150889483: loading cluster ID for follower catalog manager: success
I20260812 01:21:34.030905 14842 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f9b8bf20f70a4cd2a36e2a2150889483: acquiring CA information for follower catalog manager: success
I20260812 01:21:34.031230 14842 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f9b8bf20f70a4cd2a36e2a2150889483: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 01:21:34.078918 14896 catalog_manager.cc:1259] Loaded cluster ID: 042352d4e68844d3a36d4c6868bf30cf
I20260812 01:21:34.078998 14896 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0435323d5c1c411aa9c9163a986d5d0f: loading cluster ID for follower catalog manager: success
I20260812 01:21:34.079831 14896 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 0435323d5c1c411aa9c9163a986d5d0f: acquiring CA information for follower catalog manager: success
I20260812 01:21:34.080206 14896 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 0435323d5c1c411aa9c9163a986d5d0f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 01:21:34.167390 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 14409
I20260812 01:21:34.181499 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 14546
I20260812 01:21:34.194217 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 14671
2026-08-12T01:21:34Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3043 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786497534195564-24578-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 01:21:34 dist-test-slave-t3q3 krb5kdc[14934](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 01:21:34 dist-test-slave-t3q3 krb5kdc[14934](info): set up 2 sockets
Aug 12 01:21:34 dist-test-slave-t3q3 krb5kdc[14934](info): commencing operation
krb5kdc: starting...
W20260812 01:21:35.455248 24578 mini_kdc.cc:121] Time spent starting KDC: real 1.238s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786497534195564-24578-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786497534195564-24578-0/krb5kdc/test-user.keytab.
Aug 12 01:21:35 dist-test-slave-t3q3 krb5kdc[14934](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497695, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T01:21:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:21:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.0.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.0.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.0.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab.
Entry for principal kudu/127.24.0.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab.
I20260812 01:21:35.578608 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786497534195564-24578-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786497534195564-24578-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786497534195564-24578-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.190:41889
--webserver_interface=127.24.0.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab
--principal=kudu/127.24.0.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:44507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.0.190:41889
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:21:35.660588 14960 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:21:35.660844 14960 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:21:35.660880 14960 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:21:35.661991 14960 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:21:35.662070 14960 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:21:35.662094 14960 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:21:35.662117 14960 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:21:35.662138 14960 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:21:35.663570 14960 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:44507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786497534195564-24578-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786497534195564-24578-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.0.190:41889
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab
--principal=kudu/127.24.0.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.190:41889
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.0.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14960
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786497534195564-24578-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:21:35.663870 14960 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:21:35.664130 14960 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:21:35.669127 14975 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:21:35.669111 14971 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:21:35.669471 14960 server_base.cc:1034] running on GCE node
W20260812 01:21:35.669180 14973 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:21:35.670023 14960 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:21:35.670310 14960 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:21:35.671478 14960 hybrid_clock.cc:648] HybridClock initialized: now 1786497695671473 us; error 37 us; skew 500 ppm
Aug 12 01:21:35 dist-test-slave-t3q3 krb5kdc[14934](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497695, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:21:35.673979 14960 init.cc:377] Logged in from keytab as kudu/127.24.0.190@KRBTEST.COM (short username kudu)
I20260812 01:21:35.674458 14960 webserver.cc:460] Webserver started at http://127.24.0.190:36517/ using document root <none> and password file <none>
I20260812 01:21:35.674692 14960 fs_manager.cc:359] Metadata directory not provided
I20260812 01:21:35.674763 14960 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:21:35.674921 14960 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:21:35.675978 14960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786497534195564-24578-0/minicluster-data/master-0/data/instance:
uuid: "e0fc36b45ae84c3daf2f65acf6aeaf66"
format_stamp: "Formatted at 2026-08-12 01:21:35 on dist-test-slave-t3q3"
server_key: "2b51f4946816ea8bb0c6782b19f7536a"
server_key_iv: "72a20f62e558c6ce6e694a6415d33dd6"
server_key_version: "encryptionkey@0"
I20260812 01:21:35.676415 14960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786497534195564-24578-0/minicluster-data/master-0/wal/instance:
uuid: "e0fc36b45ae84c3daf2f65acf6aeaf66"
format_stamp: "Formatted at 2026-08-12 01:21:35 on dist-test-slave-t3q3"
server_key: "2b51f4946816ea8bb0c6782b19f7536a"
server_key_iv: "72a20f62e558c6ce6e694a6415d33dd6"
server_key_version: "encryptionkey@0"
I20260812 01:21:35.677747 14960 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:21:35.678453 14989 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:21:35.678633 14960 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 01:21:35.678742 14960 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786497534195564-24578-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786497534195564-24578-0/minicluster-data/master-0/wal
uuid: "e0fc36b45ae84c3daf2f65acf6aeaf66"
format_stamp: "Formatted at 2026-08-12 01:21:35 on dist-test-slave-t3q3"
server_key: "2b51f4946816ea8bb0c6782b19f7536a"
server_key_iv: "72a20f62e558c6ce6e694a6415d33dd6"
server_key_version: "encryptionkey@0"
I20260812 01:21:35.678815 14960 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786497534195564-24578-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786497534195564-24578-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786497534195564-24578-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:21:35.716272 14960 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:21:35.718526 14960 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:21:35.718699 14960 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:21:35.723090 14960 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.190:41889
I20260812 01:21:35.723131 15079 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.190:41889 every 8 connection(s)
I20260812 01:21:35.723551 14960 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
I20260812 01:21:35.724182 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 14960
I20260812 01:21:35.724272 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786497534195564-24578-0/minicluster-data/master-0/wal/instance
I20260812 01:21:35.724504 24578 external_mini_cluster.cc:1428] Setting key 017bdebe423cc0a19aec520133dd7940
I20260812 01:21:35.724720 15081 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:21:35.728188 15081 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0fc36b45ae84c3daf2f65acf6aeaf66: Bootstrap starting.
I20260812 01:21:35.728968 15081 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e0fc36b45ae84c3daf2f65acf6aeaf66: Neither blocks nor log segments found. Creating new log.
I20260812 01:21:35.729287 15081 log.cc:824] T 00000000000000000000000000000000 P e0fc36b45ae84c3daf2f65acf6aeaf66: Log is configured to *not* fsync() on all Append() calls
I20260812 01:21:35.730086 15081 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0fc36b45ae84c3daf2f65acf6aeaf66: No bootstrap required, opened a new log
Aug 12 01:21:35 dist-test-slave-t3q3 krb5kdc[14934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497695, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
I20260812 01:21:35.732542 15081 raft_consensus.cc:348] T 00000000000000000000000000000000 P e0fc36b45ae84c3daf2f65acf6aeaf66 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0fc36b45ae84c3daf2f65acf6aeaf66" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 41889 } }
I20260812 01:21:35.732724 15081 raft_consensus.cc:374] T 00000000000000000000000000000000 P e0fc36b45ae84c3daf2f65acf6aeaf66 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:21:35.732810 15081 raft_consensus.cc:729] T 00000000000000000000000000000000 P e0fc36b45ae84c3daf2f65acf6aeaf66 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0fc36b45ae84c3daf2f65acf6aeaf66, State: Initialized, Role: FOLLOWER
I20260812 01:21:35.732993 15081 consensus_queue.cc:260] T 00000000000000000000000000000000 P e0fc36b45ae84c3daf2f65acf6aeaf66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0fc36b45ae84c3daf2f65acf6aeaf66" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 41889 } }
I20260812 01:21:35.733100 15081 raft_consensus.cc:388] T 00000000000000000000000000000000 P e0fc36b45ae84c3daf2f65acf6aeaf66 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:21:35.733147 15081 raft_consensus.cc:482] T 00000000000000000000000000000000 P e0fc36b45ae84c3daf2f65acf6aeaf66 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:21:35.733198 15081 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e0fc36b45ae84c3daf2f65acf6aeaf66 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:21:35.733810 15081 raft_consensus.cc:504] T 00000000000000000000000000000000 P e0fc36b45ae84c3daf2f65acf6aeaf66 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0fc36b45ae84c3daf2f65acf6aeaf66" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 41889 } }
I20260812 01:21:35.733907 15081 leader_election.cc:302] T 00000000000000000000000000000000 P e0fc36b45ae84c3daf2f65acf6aeaf66 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e0fc36b45ae84c3daf2f65acf6aeaf66; no voters:
I20260812 01:21:35.734104 15081 leader_election.cc:288] T 00000000000000000000000000000000 P e0fc36b45ae84c3daf2f65acf6aeaf66 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 01:21:35.734172 15096 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e0fc36b45ae84c3daf2f65acf6aeaf66 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:21:35.734397 15096 raft_consensus.cc:686] T 00000000000000000000000000000000 P e0fc36b45ae84c3daf2f65acf6aeaf66 [term 1 LEADER]: Becoming Leader. State: Replica: e0fc36b45ae84c3daf2f65acf6aeaf66, State: Running, Role: LEADER
I20260812 01:21:35.734578 15096 consensus_queue.cc:237] T 00000000000000000000000000000000 P e0fc36b45ae84c3daf2f65acf6aeaf66 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0fc36b45ae84c3daf2f65acf6aeaf66" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 41889 } }
I20260812 01:21:35.734943 15096 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0fc36b45ae84c3daf2f65acf6aeaf66 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e0fc36b45ae84c3daf2f65acf6aeaf66. Latest consensus state: current_term: 1 leader_uuid: "e0fc36b45ae84c3daf2f65acf6aeaf66" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0fc36b45ae84c3daf2f65acf6aeaf66" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 41889 } } }
I20260812 01:21:35.735041 15096 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0fc36b45ae84c3daf2f65acf6aeaf66 [sys.catalog]: This master's current role is: LEADER
I20260812 01:21:35.735001 15097 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0fc36b45ae84c3daf2f65acf6aeaf66 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e0fc36b45ae84c3daf2f65acf6aeaf66" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0fc36b45ae84c3daf2f65acf6aeaf66" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 41889 } } }
I20260812 01:21:35.735107 15097 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0fc36b45ae84c3daf2f65acf6aeaf66 [sys.catalog]: This master's current role is: LEADER
I20260812 01:21:35.734396 15081 sys_catalog.cc:564] T 00000000000000000000000000000000 P e0fc36b45ae84c3daf2f65acf6aeaf66 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:21:35.735248 15102 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:21:35.735780 15102 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:21:35.736598 15092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:35.725252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44558 (local address 127.24.0.190:41889)
0812 01:21:35.725457 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0812 01:21:35.725464 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 01:21:35.725489 (+ 25us) server_negotiation.cc:407] Connection header received
0812 01:21:35.725890 (+ 401us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:21:35.725896 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:21:35.725942 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:21:35.726051 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:21:35.726877 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:35.727790 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:35.728849 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:35.729294 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:35.729411 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:35.732221 (+ 2810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:21:35.732225 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:21:35.732225 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:21:35.732235 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:21:35.734250 (+ 2015us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:21:35.734727 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:21:35.734729 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:21:35.734730 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:21:35.734835 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:21:35.735120 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:21:35.735121 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:21:35.735121 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:21:35.735452 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:21:35.735593 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0812 01:21:35.736200 (+ 607us) server_negotiation.cc:299] Negotiation successful
0812 01:21:35.736368 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":117,"threads_started":1}
I20260812 01:21:35.737530 15102 catalog_manager.cc:1347] Generated new cluster ID: 5451c2bb2d3f418a85dfedd47d8c251b
I20260812 01:21:35.737581 15102 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 01:21:35.737659 24578 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 01:21:35.761809 15102 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:21:35.762379 15102 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:21:35.769378 15102 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e0fc36b45ae84c3daf2f65acf6aeaf66: Generated new TSK 0
I20260812 01:21:35.769590 15102 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 01:21:35.791538 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 14960
2026-08-12T01:21:35Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1601 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 01:21:35 dist-test-slave-t3q3 krb5kdc[15138](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 01:21:35 dist-test-slave-t3q3 krb5kdc[15138](info): set up 2 sockets
Aug 12 01:21:35 dist-test-slave-t3q3 krb5kdc[15138](info): commencing operation
krb5kdc: starting...
W20260812 01:21:37.060729 24578 mini_kdc.cc:121] Time spent starting KDC: real 1.242s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/krb5kdc/test-user.keytab.
Aug 12 01:21:37 dist-test-slave-t3q3 krb5kdc[15138](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497697, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T01:21:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:21:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.0.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.0.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.0.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab.
Entry for principal kudu/127.24.0.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab.
I20260812 01:21:37.182930 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.190:38787
--webserver_interface=127.24.0.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab
--principal=kudu/127.24.0.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:33767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.0.190:38787
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:21:37.265841 15157 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:21:37.266047 15157 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:21:37.266069 15157 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:21:37.267129 15157 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:21:37.267163 15157 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:21:37.267179 15157 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:21:37.267190 15157 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:21:37.267202 15157 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:21:37.268747 15157 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:33767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.0.190:38787
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab
--principal=kudu/127.24.0.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.190:38787
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.0.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15157
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:21:37.268941 15157 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:21:37.269109 15157 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:21:37.274091 15168 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:21:37.274123 15167 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:21:37.274121 15172 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:21:37.274299 15157 server_base.cc:1034] running on GCE node
I20260812 01:21:37.274582 15157 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:21:37.274871 15157 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:21:37.276019 15157 hybrid_clock.cc:648] HybridClock initialized: now 1786497697275999 us; error 37 us; skew 500 ppm
Aug 12 01:21:37 dist-test-slave-t3q3 krb5kdc[15138](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497697, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:21:37.278678 15157 init.cc:377] Logged in from keytab as kudu/127.24.0.190@KRBTEST.COM (short username kudu)
I20260812 01:21:37.279131 15157 webserver.cc:460] Webserver started at http://127.24.0.190:45909/ using document root <none> and password file <none>
I20260812 01:21:37.279356 15157 fs_manager.cc:359] Metadata directory not provided
I20260812 01:21:37.279403 15157 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:21:37.279573 15157 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:21:37.280719 15157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/master-0/data/instance:
uuid: "eea6b06caffa4bfc8b5524bfacd0c7d3"
format_stamp: "Formatted at 2026-08-12 01:21:37 on dist-test-slave-t3q3"
server_key: "f819090b3b06392cede4af5a071fe241"
server_key_iv: "36a9b49b4a3a8d53137fed665012b789"
server_key_version: "encryptionkey@0"
I20260812 01:21:37.281092 15157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/master-0/wal/instance:
uuid: "eea6b06caffa4bfc8b5524bfacd0c7d3"
format_stamp: "Formatted at 2026-08-12 01:21:37 on dist-test-slave-t3q3"
server_key: "f819090b3b06392cede4af5a071fe241"
server_key_iv: "36a9b49b4a3a8d53137fed665012b789"
server_key_version: "encryptionkey@0"
I20260812 01:21:37.282409 15157 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 01:21:37.283098 15181 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:21:37.283289 15157 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 01:21:37.283380 15157 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/master-0/wal
uuid: "eea6b06caffa4bfc8b5524bfacd0c7d3"
format_stamp: "Formatted at 2026-08-12 01:21:37 on dist-test-slave-t3q3"
server_key: "f819090b3b06392cede4af5a071fe241"
server_key_iv: "36a9b49b4a3a8d53137fed665012b789"
server_key_version: "encryptionkey@0"
I20260812 01:21:37.283454 15157 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:21:37.299533 15157 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:21:37.301832 15157 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:21:37.301995 15157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:21:37.306397 15157 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.190:38787
I20260812 01:21:37.306430 15265 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.190:38787 every 8 connection(s)
I20260812 01:21:37.306789 15157 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
I20260812 01:21:37.307756 15266 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:21:37.307983 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 15157
I20260812 01:21:37.308118 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/master-0/wal/instance
I20260812 01:21:37.308365 24578 external_mini_cluster.cc:1428] Setting key d2332321112c1306c7ce85702d35c86b
I20260812 01:21:37.311282 15266 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eea6b06caffa4bfc8b5524bfacd0c7d3: Bootstrap starting.
I20260812 01:21:37.311957 15266 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eea6b06caffa4bfc8b5524bfacd0c7d3: Neither blocks nor log segments found. Creating new log.
I20260812 01:21:37.312304 15266 log.cc:824] T 00000000000000000000000000000000 P eea6b06caffa4bfc8b5524bfacd0c7d3: Log is configured to *not* fsync() on all Append() calls
I20260812 01:21:37.313054 15266 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eea6b06caffa4bfc8b5524bfacd0c7d3: No bootstrap required, opened a new log
Aug 12 01:21:37 dist-test-slave-t3q3 krb5kdc[15138](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497697, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
I20260812 01:21:37.315596 15266 raft_consensus.cc:348] T 00000000000000000000000000000000 P eea6b06caffa4bfc8b5524bfacd0c7d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eea6b06caffa4bfc8b5524bfacd0c7d3" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 38787 } }
I20260812 01:21:37.315752 15266 raft_consensus.cc:374] T 00000000000000000000000000000000 P eea6b06caffa4bfc8b5524bfacd0c7d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:21:37.315804 15266 raft_consensus.cc:729] T 00000000000000000000000000000000 P eea6b06caffa4bfc8b5524bfacd0c7d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eea6b06caffa4bfc8b5524bfacd0c7d3, State: Initialized, Role: FOLLOWER
I20260812 01:21:37.315960 15266 consensus_queue.cc:260] T 00000000000000000000000000000000 P eea6b06caffa4bfc8b5524bfacd0c7d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eea6b06caffa4bfc8b5524bfacd0c7d3" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 38787 } }
I20260812 01:21:37.316078 15266 raft_consensus.cc:388] T 00000000000000000000000000000000 P eea6b06caffa4bfc8b5524bfacd0c7d3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:21:37.316118 15266 raft_consensus.cc:482] T 00000000000000000000000000000000 P eea6b06caffa4bfc8b5524bfacd0c7d3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:21:37.316165 15266 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eea6b06caffa4bfc8b5524bfacd0c7d3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:21:37.316918 15266 raft_consensus.cc:504] T 00000000000000000000000000000000 P eea6b06caffa4bfc8b5524bfacd0c7d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eea6b06caffa4bfc8b5524bfacd0c7d3" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 38787 } }
I20260812 01:21:37.317039 15266 leader_election.cc:302] T 00000000000000000000000000000000 P eea6b06caffa4bfc8b5524bfacd0c7d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eea6b06caffa4bfc8b5524bfacd0c7d3; no voters:
I20260812 01:21:37.317206 15266 leader_election.cc:288] T 00000000000000000000000000000000 P eea6b06caffa4bfc8b5524bfacd0c7d3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 01:21:37.317267 15272 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eea6b06caffa4bfc8b5524bfacd0c7d3 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:21:37.317471 15266 sys_catalog.cc:564] T 00000000000000000000000000000000 P eea6b06caffa4bfc8b5524bfacd0c7d3 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:21:37.317472 15272 raft_consensus.cc:686] T 00000000000000000000000000000000 P eea6b06caffa4bfc8b5524bfacd0c7d3 [term 1 LEADER]: Becoming Leader. State: Replica: eea6b06caffa4bfc8b5524bfacd0c7d3, State: Running, Role: LEADER
I20260812 01:21:37.318125 15272 consensus_queue.cc:237] T 00000000000000000000000000000000 P eea6b06caffa4bfc8b5524bfacd0c7d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eea6b06caffa4bfc8b5524bfacd0c7d3" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 38787 } }
I20260812 01:21:37.318771 15274 sys_catalog.cc:455] T 00000000000000000000000000000000 P eea6b06caffa4bfc8b5524bfacd0c7d3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eea6b06caffa4bfc8b5524bfacd0c7d3. Latest consensus state: current_term: 1 leader_uuid: "eea6b06caffa4bfc8b5524bfacd0c7d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eea6b06caffa4bfc8b5524bfacd0c7d3" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 38787 } } }
I20260812 01:21:37.319046 15274 sys_catalog.cc:458] T 00000000000000000000000000000000 P eea6b06caffa4bfc8b5524bfacd0c7d3 [sys.catalog]: This master's current role is: LEADER
I20260812 01:21:37.318925 15273 sys_catalog.cc:455] T 00000000000000000000000000000000 P eea6b06caffa4bfc8b5524bfacd0c7d3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eea6b06caffa4bfc8b5524bfacd0c7d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eea6b06caffa4bfc8b5524bfacd0c7d3" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 38787 } } }
I20260812 01:21:37.319254 15273 sys_catalog.cc:458] T 00000000000000000000000000000000 P eea6b06caffa4bfc8b5524bfacd0c7d3 [sys.catalog]: This master's current role is: LEADER
W20260812 01:21:37.319363 15293 catalog_manager.cc:1558] T 00000000000000000000000000000000 P eea6b06caffa4bfc8b5524bfacd0c7d3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 01:21:37.319414 15293 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 01:21:37.319574 15294 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:21:37.319775 15294 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:21:37.319918 15270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:37.308917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54822 (local address 127.24.0.190:38787)
0812 01:21:37.309102 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0812 01:21:37.309118 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 01:21:37.309144 (+ 26us) server_negotiation.cc:407] Connection header received
0812 01:21:37.309597 (+ 453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:21:37.309603 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:21:37.309653 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:21:37.309769 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:21:37.310584 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:37.311761 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:37.312795 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:37.313267 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:37.313350 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:37.315919 (+ 2569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:21:37.315923 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:21:37.315924 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:21:37.315935 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:21:37.317936 (+ 2001us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:21:37.318324 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:21:37.318327 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:21:37.318327 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:21:37.318459 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:21:37.318683 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:21:37.318685 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:21:37.318685 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:21:37.319011 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:21:37.319166 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0812 01:21:37.319634 (+ 468us) server_negotiation.cc:299] Negotiation successful
0812 01:21:37.319783 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":103,"threads_started":1}
I20260812 01:21:37.321450 15294 catalog_manager.cc:1347] Generated new cluster ID: 5434671b92c84d6294600128656e9035
I20260812 01:21:37.321520 15294 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 01:21:37.329901 15294 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:21:37.330448 15294 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:21:37.334151 15294 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eea6b06caffa4bfc8b5524bfacd0c7d3: Generated new TSK 0
I20260812 01:21:37.334339 15294 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.0.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.0.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.0.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.129.keytab.
Entry for principal kudu/127.24.0.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.129.keytab.
I20260812 01:21:37.362658 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.0.129:0
--local_ip_for_outbound_sockets=127.24.0.129
--webserver_interface=127.24.0.129
--webserver_port=0
--tserver_master_addrs=127.24.0.190:38787
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.129.keytab
--principal=kudu/127.24.0.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:33767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:21:37.450821 15301 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:21:37.451026 15301 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:21:37.451050 15301 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:21:37.452229 15301 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:21:37.452272 15301 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:21:37.452298 15301 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.0.129
I20260812 01:21:37.453796 15301 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:33767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.129.keytab
--principal=kudu/127.24.0.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.0.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.0.129
--webserver_port=0
--tserver_master_addrs=127.24.0.190:38787
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15301
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.0.129
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:21:37.454052 15301 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:21:37.454308 15301 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:21:37.459432 15318 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:21:37.459393 15314 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:21:37.459542 15301 server_base.cc:1034] running on GCE node
W20260812 01:21:37.459370 15315 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:21:37.459894 15301 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:21:37.460212 15301 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:21:37.461362 15301 hybrid_clock.cc:648] HybridClock initialized: now 1786497697461339 us; error 39 us; skew 500 ppm
Aug 12 01:21:37 dist-test-slave-t3q3 krb5kdc[15138](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497697, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:21:37.463945 15301 init.cc:377] Logged in from keytab as kudu/127.24.0.129@KRBTEST.COM (short username kudu)
I20260812 01:21:37.464421 15301 webserver.cc:460] Webserver started at http://127.24.0.129:33399/ using document root <none> and password file <none>
I20260812 01:21:37.464673 15301 fs_manager.cc:359] Metadata directory not provided
I20260812 01:21:37.464742 15301 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:21:37.464890 15301 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:21:37.465927 15301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-0/data/instance:
uuid: "41ce5ec82b034993aaa0a6e2617407a7"
format_stamp: "Formatted at 2026-08-12 01:21:37 on dist-test-slave-t3q3"
server_key: "2703cfff634566d3398a91cf0eb6bd6c"
server_key_iv: "f00c7bc63e81c634d05b20bbe7f55b33"
server_key_version: "encryptionkey@0"
I20260812 01:21:37.466288 15301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-0/wal/instance:
uuid: "41ce5ec82b034993aaa0a6e2617407a7"
format_stamp: "Formatted at 2026-08-12 01:21:37 on dist-test-slave-t3q3"
server_key: "2703cfff634566d3398a91cf0eb6bd6c"
server_key_iv: "f00c7bc63e81c634d05b20bbe7f55b33"
server_key_version: "encryptionkey@0"
I20260812 01:21:37.467592 15301 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 01:21:37.468377 15332 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:21:37.468538 15301 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:21:37.468647 15301 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-0/wal
uuid: "41ce5ec82b034993aaa0a6e2617407a7"
format_stamp: "Formatted at 2026-08-12 01:21:37 on dist-test-slave-t3q3"
server_key: "2703cfff634566d3398a91cf0eb6bd6c"
server_key_iv: "f00c7bc63e81c634d05b20bbe7f55b33"
server_key_version: "encryptionkey@0"
I20260812 01:21:37.468720 15301 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:21:37.477026 15301 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:21:37.478822 15301 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:21:37.478979 15301 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:21:37.479197 15301 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:21:37.479537 15301 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:21:37.479590 15301 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:21:37.479629 15301 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:21:37.479660 15301 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:21:37.485627 15301 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.129:35121
I20260812 01:21:37.485680 15524 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.129:35121 every 8 connection(s)
I20260812 01:21:37.485991 15301 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-0/data/info.pb
I20260812 01:21:37.488276 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 15301
I20260812 01:21:37.488351 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-0/wal/instance
I20260812 01:21:37.488516 24578 external_mini_cluster.cc:1428] Setting key 0d29e5d5496f4cf913a0bbe5249c9746
Aug 12 01:21:37 dist-test-slave-t3q3 krb5kdc[15138](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497697, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.129@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
I20260812 01:21:37.495802 15270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:37.486611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.129:46397 (local address 127.24.0.190:38787)
0812 01:21:37.486661 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 01:21:37.486663 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:21:37.486870 (+ 207us) server_negotiation.cc:407] Connection header received
0812 01:21:37.487470 (+ 600us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:21:37.487471 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:21:37.487484 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:21:37.487556 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:21:37.488670 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:37.489178 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:37.490184 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:37.490399 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:37.490477 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:37.492876 (+ 2399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:21:37.492877 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:21:37.492877 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:21:37.492884 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:21:37.494284 (+ 1400us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:21:37.494803 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:21:37.494804 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:21:37.494804 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:21:37.494854 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:21:37.495147 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:21:37.495148 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:21:37.495148 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:21:37.495249 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:21:37.495301 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 01:21:37.495672 (+ 371us) server_negotiation.cc:299] Negotiation successful
0812 01:21:37.495738 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260812 01:21:37.496096 15531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:37.486581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:38787 (local address 127.24.0.129:46397)
0812 01:21:37.486761 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:37.486792 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0812 01:21:37.487363 (+ 571us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:21:37.487625 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:21:37.487629 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:21:37.488283 (+ 654us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:21:37.488526 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:21:37.488531 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:21:37.489278 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:21:37.489279 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:21:37.490088 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:21:37.490090 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:21:37.490485 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:21:37.490486 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:21:37.490628 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:37.491128 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:21:37.491144 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:21:37.492770 (+ 1626us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:21:37.494370 (+ 1600us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:21:37.494377 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:21:37.494379 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:21:37.494725 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:21:37.494925 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:21:37.494926 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:21:37.494926 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:21:37.495072 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:21:37.495319 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:21:37.495326 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:21:37.495496 (+ 170us) client_negotiation.cc:769] Sending connection context
0812 01:21:37.495652 (+ 156us) client_negotiation.cc:240] Negotiation successful
0812 01:21:37.495836 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":92,"threads_started":1}
I20260812 01:21:37.496729 15529 heartbeater.cc:347] Connected to a master server at 127.24.0.190:38787
I20260812 01:21:37.496832 15529 heartbeater.cc:464] Registering TS with master...
I20260812 01:21:37.497023 15529 heartbeater.cc:511] Master 127.24.0.190:38787 requested a full tablet report, sending...
I20260812 01:21:37.497524 15210 ts_manager.cc:194] Registered new tserver with Master: 41ce5ec82b034993aaa0a6e2617407a7 (127.24.0.129:35121)
WARNING: no policy specified for kudu/127.24.0.130@KRBTEST.COM; defaulting to no policy
I20260812 01:21:37.498137 15210 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.0.129@KRBTEST.COM'} at 127.24.0.129:46397
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.0.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.0.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.130.keytab.
Entry for principal kudu/127.24.0.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.130.keytab.
I20260812 01:21:37.514719 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.0.130:0
--local_ip_for_outbound_sockets=127.24.0.130
--webserver_interface=127.24.0.130
--webserver_port=0
--tserver_master_addrs=127.24.0.190:38787
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.130.keytab
--principal=kudu/127.24.0.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:33767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:21:37.595562 15536 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:21:37.595769 15536 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:21:37.595793 15536 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:21:37.596951 15536 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:21:37.596993 15536 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:21:37.597047 15536 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.0.130
I20260812 01:21:37.598482 15536 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:33767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.130.keytab
--principal=kudu/127.24.0.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.0.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.0.130
--webserver_port=0
--tserver_master_addrs=127.24.0.190:38787
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15536
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.0.130
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:21:37.598743 15536 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:21:37.598977 15536 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:21:37.603864 15552 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:21:37.603991 15536 server_base.cc:1034] running on GCE node
W20260812 01:21:37.603873 15554 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:21:37.604105 15548 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:21:37.604421 15536 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:21:37.604668 15536 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:21:37.605813 15536 hybrid_clock.cc:648] HybridClock initialized: now 1786497697605792 us; error 36 us; skew 500 ppm
Aug 12 01:21:37 dist-test-slave-t3q3 krb5kdc[15138](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497697, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:21:37.608284 15536 init.cc:377] Logged in from keytab as kudu/127.24.0.130@KRBTEST.COM (short username kudu)
I20260812 01:21:37.608712 15536 webserver.cc:460] Webserver started at http://127.24.0.130:40319/ using document root <none> and password file <none>
I20260812 01:21:37.608946 15536 fs_manager.cc:359] Metadata directory not provided
I20260812 01:21:37.608991 15536 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:21:37.609141 15536 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:21:37.610237 15536 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-1/data/instance:
uuid: "0748ca23dafe4fc88570234338ffdec1"
format_stamp: "Formatted at 2026-08-12 01:21:37 on dist-test-slave-t3q3"
server_key: "5f044f54d62149cb82904befdb3ad5cb"
server_key_iv: "5fd9fa8d22942fcd892eee45cda85e51"
server_key_version: "encryptionkey@0"
I20260812 01:21:37.610589 15536 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-1/wal/instance:
uuid: "0748ca23dafe4fc88570234338ffdec1"
format_stamp: "Formatted at 2026-08-12 01:21:37 on dist-test-slave-t3q3"
server_key: "5f044f54d62149cb82904befdb3ad5cb"
server_key_iv: "5fd9fa8d22942fcd892eee45cda85e51"
server_key_version: "encryptionkey@0"
I20260812 01:21:37.611871 15536 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260812 01:21:37.612774 15570 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:21:37.613018 15536 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 01:21:37.613133 15536 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-1/wal
uuid: "0748ca23dafe4fc88570234338ffdec1"
format_stamp: "Formatted at 2026-08-12 01:21:37 on dist-test-slave-t3q3"
server_key: "5f044f54d62149cb82904befdb3ad5cb"
server_key_iv: "5fd9fa8d22942fcd892eee45cda85e51"
server_key_version: "encryptionkey@0"
I20260812 01:21:37.613210 15536 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:21:37.624341 15536 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:21:37.626116 15536 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:21:37.626272 15536 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:21:37.626519 15536 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:21:37.626855 15536 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:21:37.626889 15536 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:21:37.626910 15536 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:21:37.626955 15536 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:21:37.632822 15536 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.130:35331
I20260812 01:21:37.632879 15760 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.130:35331 every 8 connection(s)
I20260812 01:21:37.633198 15536 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-1/data/info.pb
Aug 12 01:21:37 dist-test-slave-t3q3 krb5kdc[15138](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497697, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.130@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
I20260812 01:21:37.639571 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 15536
I20260812 01:21:37.639669 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-1/wal/instance
I20260812 01:21:37.639873 24578 external_mini_cluster.cc:1428] Setting key 752e657efc0b63e1a8ba61c5f110ffe1
I20260812 01:21:37.642949 15270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:37.633912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.130:43227 (local address 127.24.0.190:38787)
0812 01:21:37.633958 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 01:21:37.633962 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 01:21:37.634180 (+ 218us) server_negotiation.cc:407] Connection header received
0812 01:21:37.634715 (+ 535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:21:37.634717 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:21:37.634730 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:21:37.634783 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:21:37.635733 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:37.636293 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:37.637283 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:37.637480 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:37.637575 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:37.639573 (+ 1998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:21:37.639575 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:21:37.639575 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:21:37.639583 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:21:37.641549 (+ 1966us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:21:37.641940 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:21:37.641942 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:21:37.641942 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:21:37.642019 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:21:37.642268 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:21:37.642269 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:21:37.642270 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:21:37.642418 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:21:37.642489 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 01:21:37.642774 (+ 285us) server_negotiation.cc:299] Negotiation successful
0812 01:21:37.642856 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 01:21:37.643163 15765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:37.633876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:38787 (local address 127.24.0.130:43227)
0812 01:21:37.634055 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:37.634090 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0812 01:21:37.634604 (+ 514us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:21:37.634852 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:21:37.634857 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:21:37.635436 (+ 579us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:21:37.635621 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:21:37.635626 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:21:37.636385 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:21:37.636386 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:21:37.637185 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:21:37.637187 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:21:37.637586 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:21:37.637587 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:21:37.637722 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:37.638129 (+ 407us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:21:37.638137 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:21:37.639466 (+ 1329us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:21:37.641638 (+ 2172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:21:37.641645 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:21:37.641646 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:21:37.641865 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:21:37.642085 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:21:37.642086 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:21:37.642086 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:21:37.642203 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:21:37.642483 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:21:37.642488 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:21:37.642598 (+ 110us) client_negotiation.cc:769] Sending connection context
0812 01:21:37.642746 (+ 148us) client_negotiation.cc:240] Negotiation successful
0812 01:21:37.642908 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":74,"threads_started":1}
I20260812 01:21:37.643656 15762 heartbeater.cc:347] Connected to a master server at 127.24.0.190:38787
I20260812 01:21:37.643743 15762 heartbeater.cc:464] Registering TS with master...
I20260812 01:21:37.643929 15762 heartbeater.cc:511] Master 127.24.0.190:38787 requested a full tablet report, sending...
I20260812 01:21:37.644331 15210 ts_manager.cc:194] Registered new tserver with Master: 0748ca23dafe4fc88570234338ffdec1 (127.24.0.130:35331)
I20260812 01:21:37.644716 15210 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.0.130@KRBTEST.COM'} at 127.24.0.130:43227
WARNING: no policy specified for kudu/127.24.0.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.0.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.0.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.131.keytab.
Entry for principal kudu/127.24.0.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.131.keytab.
I20260812 01:21:37.666637 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.0.131:0
--local_ip_for_outbound_sockets=127.24.0.131
--webserver_interface=127.24.0.131
--webserver_port=0
--tserver_master_addrs=127.24.0.190:38787
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.131.keytab
--principal=kudu/127.24.0.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:33767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:21:37.750891 15768 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:21:37.751097 15768 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:21:37.751121 15768 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:21:37.752316 15768 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:21:37.752359 15768 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:21:37.752385 15768 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.0.131
I20260812 01:21:37.753774 15768 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:33767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.131.keytab
--principal=kudu/127.24.0.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.0.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.0.131
--webserver_port=0
--tserver_master_addrs=127.24.0.190:38787
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15768
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.0.131
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:21:37.753957 15768 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:21:37.754138 15768 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:21:37.759230 15786 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:21:37.759218 15783 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:21:37.759428 15768 server_base.cc:1034] running on GCE node
W20260812 01:21:37.759218 15782 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:21:37.759716 15768 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:21:37.759984 15768 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:21:37.761147 15768 hybrid_clock.cc:648] HybridClock initialized: now 1786497697761151 us; error 48 us; skew 500 ppm
Aug 12 01:21:37 dist-test-slave-t3q3 krb5kdc[15138](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497697, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:21:37.763582 15768 init.cc:377] Logged in from keytab as kudu/127.24.0.131@KRBTEST.COM (short username kudu)
I20260812 01:21:37.764091 15768 webserver.cc:460] Webserver started at http://127.24.0.131:42015/ using document root <none> and password file <none>
I20260812 01:21:37.764328 15768 fs_manager.cc:359] Metadata directory not provided
I20260812 01:21:37.764397 15768 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:21:37.764554 15768 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:21:37.765561 15768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-2/data/instance:
uuid: "0d48ccc189c44aa59f80d12affe2a109"
format_stamp: "Formatted at 2026-08-12 01:21:37 on dist-test-slave-t3q3"
server_key: "4e8e79c992c5758da2ada0484c88e2bb"
server_key_iv: "b1bd463be88d537e77a43a1b4ef41879"
server_key_version: "encryptionkey@0"
I20260812 01:21:37.765937 15768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-2/wal/instance:
uuid: "0d48ccc189c44aa59f80d12affe2a109"
format_stamp: "Formatted at 2026-08-12 01:21:37 on dist-test-slave-t3q3"
server_key: "4e8e79c992c5758da2ada0484c88e2bb"
server_key_iv: "b1bd463be88d537e77a43a1b4ef41879"
server_key_version: "encryptionkey@0"
I20260812 01:21:37.767246 15768 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 01:21:37.767964 15795 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:21:37.768167 15768 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 01:21:37.768268 15768 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-2/wal
uuid: "0d48ccc189c44aa59f80d12affe2a109"
format_stamp: "Formatted at 2026-08-12 01:21:37 on dist-test-slave-t3q3"
server_key: "4e8e79c992c5758da2ada0484c88e2bb"
server_key_iv: "b1bd463be88d537e77a43a1b4ef41879"
server_key_version: "encryptionkey@0"
I20260812 01:21:37.768355 15768 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:21:37.776808 15768 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:21:37.778497 15768 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:21:37.778658 15768 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:21:37.778872 15768 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:21:37.779168 15768 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:21:37.779223 15768 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:21:37.779263 15768 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:21:37.779290 15768 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:21:37.785190 15768 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.131:43669
I20260812 01:21:37.785228 16010 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.131:43669 every 8 connection(s)
I20260812 01:21:37.785534 15768 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-2/data/info.pb
Aug 12 01:21:37 dist-test-slave-t3q3 krb5kdc[15138](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497697, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.131@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
I20260812 01:21:37.792220 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 15768
I20260812 01:21:37.792325 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-2/wal/instance
I20260812 01:21:37.792537 24578 external_mini_cluster.cc:1428] Setting key 64a453e3b8ef5fa788878a6266a2c891
I20260812 01:21:37.794602 15270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:37.786222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.131:47877 (local address 127.24.0.190:38787)
0812 01:21:37.786283 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 01:21:37.786285 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:21:37.786546 (+ 261us) server_negotiation.cc:407] Connection header received
0812 01:21:37.786994 (+ 448us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:21:37.786995 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:21:37.787005 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:21:37.787103 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:21:37.788092 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:37.788581 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:37.789564 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:37.789762 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:37.789838 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:37.791802 (+ 1964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:21:37.791803 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:21:37.791803 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:21:37.791809 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:21:37.793377 (+ 1568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:21:37.793860 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:21:37.793861 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:21:37.793861 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:21:37.793912 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:21:37.794096 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:21:37.794096 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:21:37.794096 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:21:37.794206 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:21:37.794257 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 01:21:37.794483 (+ 226us) server_negotiation.cc:299] Negotiation successful
0812 01:21:37.794540 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 01:21:37.794811 16017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:37.786228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:38787 (local address 127.24.0.131:47877)
0812 01:21:37.786391 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:37.786447 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0812 01:21:37.786911 (+ 464us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:21:37.787183 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:21:37.787186 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:21:37.787787 (+ 601us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:21:37.787971 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:21:37.787975 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:21:37.788680 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:21:37.788681 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:21:37.789470 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:21:37.789472 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:21:37.789844 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:21:37.789846 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:21:37.789971 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:37.790381 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:21:37.790389 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:21:37.791718 (+ 1329us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:21:37.793555 (+ 1837us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:21:37.793561 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:21:37.793564 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:21:37.793792 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:21:37.793959 (+ 167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:21:37.793959 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:21:37.793959 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:21:37.794046 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:21:37.794265 (+ 219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:21:37.794269 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:21:37.794375 (+ 106us) client_negotiation.cc:769] Sending connection context
0812 01:21:37.794456 (+ 81us) client_negotiation.cc:240] Negotiation successful
0812 01:21:37.794570 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":88,"threads_started":1}
I20260812 01:21:37.795354 16012 heartbeater.cc:347] Connected to a master server at 127.24.0.190:38787
I20260812 01:21:37.795434 16012 heartbeater.cc:464] Registering TS with master...
I20260812 01:21:37.795564 16012 heartbeater.cc:511] Master 127.24.0.190:38787 requested a full tablet report, sending...
I20260812 01:21:37.795897 15210 ts_manager.cc:194] Registered new tserver with Master: 0d48ccc189c44aa59f80d12affe2a109 (127.24.0.131:43669)
I20260812 01:21:37.796319 15210 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.0.131@KRBTEST.COM'} at 127.24.0.131:47877
I20260812 01:21:37.804468 24578 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 01:21:37.811708 15270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:37.805847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54826 (local address 127.24.0.190:38787)
0812 01:21:37.805909 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0812 01:21:37.805911 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:21:37.806036 (+ 125us) server_negotiation.cc:407] Connection header received
0812 01:21:37.806076 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:21:37.806077 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:21:37.806085 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:21:37.806135 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:21:37.806717 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:37.807174 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:37.807878 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:37.808049 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:37.808145 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:37.809310 (+ 1165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:21:37.809311 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:21:37.809312 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:21:37.809319 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:21:37.810534 (+ 1215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:21:37.810879 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:21:37.810880 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:21:37.810881 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:21:37.810941 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:21:37.811157 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:21:37.811158 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:21:37.811158 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:21:37.811314 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:21:37.811424 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0812 01:21:37.811532 (+ 108us) server_negotiation.cc:299] Negotiation successful
0812 01:21:37.811609 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 01:21:37.813186 15210 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54826:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 01:21:37.814720 15210 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 01:21:37.821702 16028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:37.817060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.130:35331 (local address 127.0.0.1:45552)
0812 01:21:37.817227 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:37.817269 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 01:21:37.817336 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:21:37.817718 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:21:37.817721 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:21:37.817722 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:21:37.817830 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:21:37.817832 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:21:37.819346 (+ 1514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:21:37.819347 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:21:37.820473 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:21:37.820476 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:21:37.821415 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:21:37.821416 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:21:37.821478 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:37.821482 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 01:21:37.821514 (+ 32us) client_negotiation.cc:240] Negotiation successful
0812 01:21:37.821600 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"mutex_wait_us":27,"thread_start_us":91,"threads_started":1}
I20260812 01:21:37.821705 16029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:37.817229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40166 (local address 127.24.0.129:35121)
0812 01:21:37.817478 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0812 01:21:37.817483 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 01:21:37.817639 (+ 156us) server_negotiation.cc:407] Connection header received
0812 01:21:37.817947 (+ 308us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:21:37.817950 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:21:37.817967 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:21:37.818043 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:21:37.818272 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:37.819250 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:37.820739 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:37.821236 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:37.821327 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:37.821391 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 01:21:37.821577 (+ 186us) server_negotiation.cc:299] Negotiation successful
0812 01:21:37.821622 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":98,"threads_started":1}
I20260812 01:21:37.821713 16034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:37.817493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57664 (local address 127.24.0.131:43669)
0812 01:21:37.817684 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0812 01:21:37.817688 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 01:21:37.817698 (+ 10us) server_negotiation.cc:407] Connection header received
0812 01:21:37.817736 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:21:37.817738 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:21:37.817752 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:21:37.817826 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:21:37.818050 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:37.819106 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:37.820724 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:37.821388 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:37.821456 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:37.821516 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 01:21:37.821592 (+ 76us) server_negotiation.cc:299] Negotiation successful
0812 01:21:37.821636 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":79,"threads_started":1}
I20260812 01:21:37.821899 16032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:37.817280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45552 (local address 127.24.0.130:35331)
0812 01:21:37.817482 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0812 01:21:37.817487 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 01:21:37.817496 (+ 9us) server_negotiation.cc:407] Connection header received
0812 01:21:37.817538 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:21:37.817540 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:21:37.817557 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:21:37.817712 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:21:37.817897 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:37.819264 (+ 1367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:37.820574 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:37.821303 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:37.821673 (+ 370us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:37.821749 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 01:21:37.821777 (+ 28us) server_negotiation.cc:299] Negotiation successful
0812 01:21:37.821824 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":83,"threads_started":1}
I20260812 01:21:37.821920 16030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:37.817106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.129:35121 (local address 127.0.0.1:40166)
0812 01:21:37.817494 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:37.817511 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 01:21:37.817579 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:21:37.818128 (+ 549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:21:37.818128 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:21:37.818130 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:21:37.818168 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:21:37.818169 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:21:37.819365 (+ 1196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:21:37.819366 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:21:37.820635 (+ 1269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:21:37.820638 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:21:37.821341 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:21:37.821342 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:21:37.821428 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:37.821445 (+ 17us) client_negotiation.cc:769] Sending connection context
0812 01:21:37.821526 (+ 81us) client_negotiation.cc:240] Negotiation successful
0812 01:21:37.821850 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":374,"mutex_wait_us":320,"thread_start_us":73,"threads_started":1}
I20260812 01:21:37.822499 16026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:37.817302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.131:43669 (local address 127.0.0.1:57664)
0812 01:21:37.817533 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:37.817543 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 01:21:37.817587 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:21:37.817912 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:21:37.817913 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:21:37.817915 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:21:37.817969 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:21:37.817970 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:21:37.819208 (+ 1238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:21:37.819209 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:21:37.820636 (+ 1427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:21:37.820638 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:21:37.821454 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:21:37.821455 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:21:37.821522 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:37.821527 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 01:21:37.821582 (+ 55us) client_negotiation.cc:240] Negotiation successful
0812 01:21:37.821623 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"mutex_wait_us":50,"thread_start_us":44,"threads_started":1}
I20260812 01:21:37.822716 15647 tablet_service.cc:1467] Processing CreateTablet for tablet 727a26a4697e4372b04a50d827626bb7 (DEFAULT_TABLE table=test-table [id=0c910d5f128d4a21b6b824e1275cc5ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:21:37.822714 15411 tablet_service.cc:1467] Processing CreateTablet for tablet 727a26a4697e4372b04a50d827626bb7 (DEFAULT_TABLE table=test-table [id=0c910d5f128d4a21b6b824e1275cc5ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:21:37.823132 15647 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 727a26a4697e4372b04a50d827626bb7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:21:37.823136 15411 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 727a26a4697e4372b04a50d827626bb7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:21:37.823171 15893 tablet_service.cc:1467] Processing CreateTablet for tablet 727a26a4697e4372b04a50d827626bb7 (DEFAULT_TABLE table=test-table [id=0c910d5f128d4a21b6b824e1275cc5ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:21:37.823552 15893 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 727a26a4697e4372b04a50d827626bb7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:21:37.826022 16035 tablet_bootstrap.cc:492] T 727a26a4697e4372b04a50d827626bb7 P 0748ca23dafe4fc88570234338ffdec1: Bootstrap starting.
I20260812 01:21:37.826619 16040 tablet_bootstrap.cc:492] T 727a26a4697e4372b04a50d827626bb7 P 41ce5ec82b034993aaa0a6e2617407a7: Bootstrap starting.
I20260812 01:21:37.826777 16035 tablet_bootstrap.cc:654] T 727a26a4697e4372b04a50d827626bb7 P 0748ca23dafe4fc88570234338ffdec1: Neither blocks nor log segments found. Creating new log.
I20260812 01:21:37.826902 16041 tablet_bootstrap.cc:492] T 727a26a4697e4372b04a50d827626bb7 P 0d48ccc189c44aa59f80d12affe2a109: Bootstrap starting.
I20260812 01:21:37.827353 16040 tablet_bootstrap.cc:654] T 727a26a4697e4372b04a50d827626bb7 P 41ce5ec82b034993aaa0a6e2617407a7: Neither blocks nor log segments found. Creating new log.
I20260812 01:21:37.827607 16041 tablet_bootstrap.cc:654] T 727a26a4697e4372b04a50d827626bb7 P 0d48ccc189c44aa59f80d12affe2a109: Neither blocks nor log segments found. Creating new log.
I20260812 01:21:37.827636 16035 log.cc:824] T 727a26a4697e4372b04a50d827626bb7 P 0748ca23dafe4fc88570234338ffdec1: Log is configured to *not* fsync() on all Append() calls
I20260812 01:21:37.828037 16040 log.cc:824] T 727a26a4697e4372b04a50d827626bb7 P 41ce5ec82b034993aaa0a6e2617407a7: Log is configured to *not* fsync() on all Append() calls
I20260812 01:21:37.828205 16041 log.cc:824] T 727a26a4697e4372b04a50d827626bb7 P 0d48ccc189c44aa59f80d12affe2a109: Log is configured to *not* fsync() on all Append() calls
I20260812 01:21:37.828553 16035 tablet_bootstrap.cc:492] T 727a26a4697e4372b04a50d827626bb7 P 0748ca23dafe4fc88570234338ffdec1: No bootstrap required, opened a new log
I20260812 01:21:37.828635 16035 ts_tablet_manager.cc:1397] T 727a26a4697e4372b04a50d827626bb7 P 0748ca23dafe4fc88570234338ffdec1: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 01:21:37.828811 16040 tablet_bootstrap.cc:492] T 727a26a4697e4372b04a50d827626bb7 P 41ce5ec82b034993aaa0a6e2617407a7: No bootstrap required, opened a new log
I20260812 01:21:37.828892 16040 ts_tablet_manager.cc:1397] T 727a26a4697e4372b04a50d827626bb7 P 41ce5ec82b034993aaa0a6e2617407a7: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 01:21:37.829042 16041 tablet_bootstrap.cc:492] T 727a26a4697e4372b04a50d827626bb7 P 0d48ccc189c44aa59f80d12affe2a109: No bootstrap required, opened a new log
I20260812 01:21:37.829116 16041 ts_tablet_manager.cc:1397] T 727a26a4697e4372b04a50d827626bb7 P 0d48ccc189c44aa59f80d12affe2a109: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 01:21:37.830295 16040 raft_consensus.cc:348] T 727a26a4697e4372b04a50d827626bb7 P 41ce5ec82b034993aaa0a6e2617407a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0748ca23dafe4fc88570234338ffdec1" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 35331 } } peers { permanent_uuid: "41ce5ec82b034993aaa0a6e2617407a7" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 35121 } } peers { permanent_uuid: "0d48ccc189c44aa59f80d12affe2a109" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 43669 } }
I20260812 01:21:37.830490 16040 raft_consensus.cc:374] T 727a26a4697e4372b04a50d827626bb7 P 41ce5ec82b034993aaa0a6e2617407a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:21:37.830535 16040 raft_consensus.cc:729] T 727a26a4697e4372b04a50d827626bb7 P 41ce5ec82b034993aaa0a6e2617407a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41ce5ec82b034993aaa0a6e2617407a7, State: Initialized, Role: FOLLOWER
I20260812 01:21:37.830663 16040 consensus_queue.cc:260] T 727a26a4697e4372b04a50d827626bb7 P 41ce5ec82b034993aaa0a6e2617407a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0748ca23dafe4fc88570234338ffdec1" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 35331 } } peers { permanent_uuid: "41ce5ec82b034993aaa0a6e2617407a7" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 35121 } } peers { permanent_uuid: "0d48ccc189c44aa59f80d12affe2a109" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 43669 } }
I20260812 01:21:37.830631 16035 raft_consensus.cc:348] T 727a26a4697e4372b04a50d827626bb7 P 0748ca23dafe4fc88570234338ffdec1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0748ca23dafe4fc88570234338ffdec1" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 35331 } } peers { permanent_uuid: "41ce5ec82b034993aaa0a6e2617407a7" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 35121 } } peers { permanent_uuid: "0d48ccc189c44aa59f80d12affe2a109" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 43669 } }
I20260812 01:21:37.830788 16035 raft_consensus.cc:374] T 727a26a4697e4372b04a50d827626bb7 P 0748ca23dafe4fc88570234338ffdec1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:21:37.830826 16035 raft_consensus.cc:729] T 727a26a4697e4372b04a50d827626bb7 P 0748ca23dafe4fc88570234338ffdec1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0748ca23dafe4fc88570234338ffdec1, State: Initialized, Role: FOLLOWER
I20260812 01:21:37.830952 15529 heartbeater.cc:503] Master 127.24.0.190:38787 was elected leader, sending a full tablet report...
I20260812 01:21:37.830955 16035 consensus_queue.cc:260] T 727a26a4697e4372b04a50d827626bb7 P 0748ca23dafe4fc88570234338ffdec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0748ca23dafe4fc88570234338ffdec1" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 35331 } } peers { permanent_uuid: "41ce5ec82b034993aaa0a6e2617407a7" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 35121 } } peers { permanent_uuid: "0d48ccc189c44aa59f80d12affe2a109" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 43669 } }
I20260812 01:21:37.830950 16040 ts_tablet_manager.cc:1428] T 727a26a4697e4372b04a50d827626bb7 P 41ce5ec82b034993aaa0a6e2617407a7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 01:21:37.831087 16041 raft_consensus.cc:348] T 727a26a4697e4372b04a50d827626bb7 P 0d48ccc189c44aa59f80d12affe2a109 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0748ca23dafe4fc88570234338ffdec1" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 35331 } } peers { permanent_uuid: "41ce5ec82b034993aaa0a6e2617407a7" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 35121 } } peers { permanent_uuid: "0d48ccc189c44aa59f80d12affe2a109" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 43669 } }
I20260812 01:21:37.831238 16041 raft_consensus.cc:374] T 727a26a4697e4372b04a50d827626bb7 P 0d48ccc189c44aa59f80d12affe2a109 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:21:37.831277 16041 raft_consensus.cc:729] T 727a26a4697e4372b04a50d827626bb7 P 0d48ccc189c44aa59f80d12affe2a109 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d48ccc189c44aa59f80d12affe2a109, State: Initialized, Role: FOLLOWER
I20260812 01:21:37.831283 15762 heartbeater.cc:503] Master 127.24.0.190:38787 was elected leader, sending a full tablet report...
I20260812 01:21:37.831394 16041 consensus_queue.cc:260] T 727a26a4697e4372b04a50d827626bb7 P 0d48ccc189c44aa59f80d12affe2a109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0748ca23dafe4fc88570234338ffdec1" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 35331 } } peers { permanent_uuid: "41ce5ec82b034993aaa0a6e2617407a7" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 35121 } } peers { permanent_uuid: "0d48ccc189c44aa59f80d12affe2a109" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 43669 } }
I20260812 01:21:37.831506 16035 ts_tablet_manager.cc:1428] T 727a26a4697e4372b04a50d827626bb7 P 0748ca23dafe4fc88570234338ffdec1: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 01:21:37.831748 16041 ts_tablet_manager.cc:1428] T 727a26a4697e4372b04a50d827626bb7 P 0d48ccc189c44aa59f80d12affe2a109: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260812 01:21:37.831768 16012 heartbeater.cc:503] Master 127.24.0.190:38787 was elected leader, sending a full tablet report...
W20260812 01:21:37.883854 15764 tablet.cc:2369] T 727a26a4697e4372b04a50d827626bb7 P 0748ca23dafe4fc88570234338ffdec1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 01:21:37.986711 15530 tablet.cc:2369] T 727a26a4697e4372b04a50d827626bb7 P 41ce5ec82b034993aaa0a6e2617407a7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 01:21:38.036222 16013 tablet.cc:2369] T 727a26a4697e4372b04a50d827626bb7 P 0d48ccc189c44aa59f80d12affe2a109: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 01:21:38.062788 16048 raft_consensus.cc:482] T 727a26a4697e4372b04a50d827626bb7 P 0748ca23dafe4fc88570234338ffdec1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 01:21:38.062927 16048 raft_consensus.cc:504] T 727a26a4697e4372b04a50d827626bb7 P 0748ca23dafe4fc88570234338ffdec1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0748ca23dafe4fc88570234338ffdec1" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 35331 } } peers { permanent_uuid: "41ce5ec82b034993aaa0a6e2617407a7" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 35121 } } peers { permanent_uuid: "0d48ccc189c44aa59f80d12affe2a109" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 43669 } }
I20260812 01:21:38.063416 16048 leader_election.cc:288] T 727a26a4697e4372b04a50d827626bb7 P 0748ca23dafe4fc88570234338ffdec1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41ce5ec82b034993aaa0a6e2617407a7 (127.24.0.129:35121), 0d48ccc189c44aa59f80d12affe2a109 (127.24.0.131:43669)
I20260812 01:21:38.066416 16034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:38.063752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.130:49523 (local address 127.24.0.131:43669)
0812 01:21:38.063794 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0812 01:21:38.063796 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:21:38.063947 (+ 151us) server_negotiation.cc:407] Connection header received
0812 01:21:38.063976 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:21:38.063977 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:21:38.063987 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:21:38.064049 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:21:38.064267 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:38.064745 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:38.065664 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:38.066049 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:38.066143 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:38.066152 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 01:21:38.066308 (+ 156us) server_negotiation.cc:299] Negotiation successful
0812 01:21:38.066344 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 01:21:38.066416 16029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:38.063724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.130:59973 (local address 127.24.0.129:35121)
0812 01:21:38.063774 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 01:21:38.063776 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:21:38.063789 (+ 13us) server_negotiation.cc:407] Connection header received
0812 01:21:38.063873 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:21:38.063874 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:21:38.063886 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:21:38.063942 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:21:38.064134 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:38.064644 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:38.065709 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:38.066047 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:38.066136 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:38.066146 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 01:21:38.066307 (+ 161us) server_negotiation.cc:299] Negotiation successful
0812 01:21:38.066344 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260812 01:21:38.066439 15765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:38.063648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.129:35121 (local address 127.24.0.130:59973)
0812 01:21:38.063730 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:38.063743 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 01:21:38.063791 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:21:38.063963 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:21:38.063965 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:21:38.063967 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:21:38.064051 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:21:38.064053 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:21:38.064720 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:21:38.064721 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:21:38.065572 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:21:38.065574 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:21:38.066134 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:21:38.066135 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:21:38.066226 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:38.066229 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 01:21:38.066300 (+ 71us) client_negotiation.cc:240] Negotiation successful
0812 01:21:38.066344 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260812 01:21:38.066454 16050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:38.063650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.131:43669 (local address 127.24.0.130:49523)
0812 01:21:38.063843 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:38.063854 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 01:21:38.063917 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:21:38.064104 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:21:38.064105 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:21:38.064107 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:21:38.064179 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:21:38.064181 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:21:38.064831 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:21:38.064832 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:21:38.065572 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:21:38.065574 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:21:38.066134 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:21:38.066136 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:21:38.066226 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:38.066230 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 01:21:38.066299 (+ 69us) client_negotiation.cc:240] Negotiation successful
0812 01:21:38.066339 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"mutex_wait_us":18,"thread_start_us":107,"threads_started":1}
I20260812 01:21:38.066840 15452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "727a26a4697e4372b04a50d827626bb7" candidate_uuid: "0748ca23dafe4fc88570234338ffdec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41ce5ec82b034993aaa0a6e2617407a7" is_pre_election: true
I20260812 01:21:38.067019 15452 raft_consensus.cc:2454] T 727a26a4697e4372b04a50d827626bb7 P 41ce5ec82b034993aaa0a6e2617407a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0748ca23dafe4fc88570234338ffdec1 in term 0.
I20260812 01:21:38.066993 15933 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "727a26a4697e4372b04a50d827626bb7" candidate_uuid: "0748ca23dafe4fc88570234338ffdec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d48ccc189c44aa59f80d12affe2a109" is_pre_election: true
I20260812 01:21:38.067148 15933 raft_consensus.cc:2454] T 727a26a4697e4372b04a50d827626bb7 P 0d48ccc189c44aa59f80d12affe2a109 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0748ca23dafe4fc88570234338ffdec1 in term 0.
I20260812 01:21:38.067412 15572 leader_election.cc:302] T 727a26a4697e4372b04a50d827626bb7 P 0748ca23dafe4fc88570234338ffdec1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0748ca23dafe4fc88570234338ffdec1, 0d48ccc189c44aa59f80d12affe2a109; no voters:
I20260812 01:21:38.067551 16048 raft_consensus.cc:2781] T 727a26a4697e4372b04a50d827626bb7 P 0748ca23dafe4fc88570234338ffdec1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 01:21:38.067620 16048 raft_consensus.cc:482] T 727a26a4697e4372b04a50d827626bb7 P 0748ca23dafe4fc88570234338ffdec1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 01:21:38.067644 16048 raft_consensus.cc:3037] T 727a26a4697e4372b04a50d827626bb7 P 0748ca23dafe4fc88570234338ffdec1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:21:38.068418 16048 raft_consensus.cc:504] T 727a26a4697e4372b04a50d827626bb7 P 0748ca23dafe4fc88570234338ffdec1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0748ca23dafe4fc88570234338ffdec1" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 35331 } } peers { permanent_uuid: "41ce5ec82b034993aaa0a6e2617407a7" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 35121 } } peers { permanent_uuid: "0d48ccc189c44aa59f80d12affe2a109" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 43669 } }
I20260812 01:21:38.068559 16048 leader_election.cc:288] T 727a26a4697e4372b04a50d827626bb7 P 0748ca23dafe4fc88570234338ffdec1 [CANDIDATE]: Term 1 election: Requested vote from peers 41ce5ec82b034993aaa0a6e2617407a7 (127.24.0.129:35121), 0d48ccc189c44aa59f80d12affe2a109 (127.24.0.131:43669)
I20260812 01:21:38.068712 15933 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "727a26a4697e4372b04a50d827626bb7" candidate_uuid: "0748ca23dafe4fc88570234338ffdec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d48ccc189c44aa59f80d12affe2a109"
I20260812 01:21:38.068712 15452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "727a26a4697e4372b04a50d827626bb7" candidate_uuid: "0748ca23dafe4fc88570234338ffdec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41ce5ec82b034993aaa0a6e2617407a7"
I20260812 01:21:38.068791 15452 raft_consensus.cc:3037] T 727a26a4697e4372b04a50d827626bb7 P 41ce5ec82b034993aaa0a6e2617407a7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:21:38.068813 15933 raft_consensus.cc:3037] T 727a26a4697e4372b04a50d827626bb7 P 0d48ccc189c44aa59f80d12affe2a109 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:21:38.069504 15452 raft_consensus.cc:2454] T 727a26a4697e4372b04a50d827626bb7 P 41ce5ec82b034993aaa0a6e2617407a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0748ca23dafe4fc88570234338ffdec1 in term 1.
I20260812 01:21:38.069633 15933 raft_consensus.cc:2454] T 727a26a4697e4372b04a50d827626bb7 P 0d48ccc189c44aa59f80d12affe2a109 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0748ca23dafe4fc88570234338ffdec1 in term 1.
I20260812 01:21:38.069681 15571 leader_election.cc:302] T 727a26a4697e4372b04a50d827626bb7 P 0748ca23dafe4fc88570234338ffdec1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0748ca23dafe4fc88570234338ffdec1, 41ce5ec82b034993aaa0a6e2617407a7; no voters:
I20260812 01:21:38.069782 16048 raft_consensus.cc:2781] T 727a26a4697e4372b04a50d827626bb7 P 0748ca23dafe4fc88570234338ffdec1 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:21:38.069932 16048 raft_consensus.cc:686] T 727a26a4697e4372b04a50d827626bb7 P 0748ca23dafe4fc88570234338ffdec1 [term 1 LEADER]: Becoming Leader. State: Replica: 0748ca23dafe4fc88570234338ffdec1, State: Running, Role: LEADER
I20260812 01:21:38.070043 16048 consensus_queue.cc:237] T 727a26a4697e4372b04a50d827626bb7 P 0748ca23dafe4fc88570234338ffdec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0748ca23dafe4fc88570234338ffdec1" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 35331 } } peers { permanent_uuid: "41ce5ec82b034993aaa0a6e2617407a7" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 35121 } } peers { permanent_uuid: "0d48ccc189c44aa59f80d12affe2a109" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 43669 } }
I20260812 01:21:38.070945 15208 catalog_manager.cc:5495] T 727a26a4697e4372b04a50d827626bb7 P 0748ca23dafe4fc88570234338ffdec1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0748ca23dafe4fc88570234338ffdec1 (127.24.0.130). New cstate: current_term: 1 leader_uuid: "0748ca23dafe4fc88570234338ffdec1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0748ca23dafe4fc88570234338ffdec1" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 35331 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41ce5ec82b034993aaa0a6e2617407a7" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 35121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d48ccc189c44aa59f80d12affe2a109" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 43669 } health_report { overall_health: UNKNOWN } } }
I20260812 01:21:38.105301 16032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:38.102800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45562 (local address 127.24.0.130:35331)
0812 01:21:38.102869 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0812 01:21:38.102871 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:21:38.102881 (+ 10us) server_negotiation.cc:407] Connection header received
0812 01:21:38.102972 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:21:38.102973 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:21:38.102994 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:21:38.103078 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:21:38.103240 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:38.103727 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:38.104551 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:38.104745 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:38.104804 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:38.104947 (+ 143us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:21:38.105049 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:21:38.105115 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 01:21:38.105190 (+ 75us) server_negotiation.cc:299] Negotiation successful
0812 01:21:38.105216 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 01:21:38.107173 15933 raft_consensus.cc:1261] T 727a26a4697e4372b04a50d827626bb7 P 0d48ccc189c44aa59f80d12affe2a109 [term 1 FOLLOWER]: Refusing update from remote peer 0748ca23dafe4fc88570234338ffdec1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:21:38.107170 15452 raft_consensus.cc:1261] T 727a26a4697e4372b04a50d827626bb7 P 41ce5ec82b034993aaa0a6e2617407a7 [term 1 FOLLOWER]: Refusing update from remote peer 0748ca23dafe4fc88570234338ffdec1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:21:38.107560 16054 consensus_queue.cc:1035] T 727a26a4697e4372b04a50d827626bb7 P 0748ca23dafe4fc88570234338ffdec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d48ccc189c44aa59f80d12affe2a109" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 43669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:21:38.107705 16048 consensus_queue.cc:1035] T 727a26a4697e4372b04a50d827626bb7 P 0748ca23dafe4fc88570234338ffdec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41ce5ec82b034993aaa0a6e2617407a7" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 35121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:21:38.110059 16061 mvcc.cc:205] Tried to move back new op lower bound from 7317494571444228096 to 7317494571295477760. Current Snapshot: MvccSnapshot[applied={T|T < 7317494571444228096}]
I20260812 01:21:38.110571 16062 mvcc.cc:205] Tried to move back new op lower bound from 7317494571444228096 to 7317494571295477760. Current Snapshot: MvccSnapshot[applied={T|T < 7317494571444228096}]
I20260812 01:21:38.112304 15210 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54826:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:21:38.112399 15210 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54826:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:21:38.113157 15210 catalog_manager.cc:5782] T 00000000000000000000000000000000 P eea6b06caffa4bfc8b5524bfacd0c7d3: Sending DeleteTablet for 3 replicas of tablet 727a26a4697e4372b04a50d827626bb7
I20260812 01:21:38.113516 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 15301
I20260812 01:21:38.113533 15411 tablet_service.cc:1514] Processing DeleteTablet for tablet 727a26a4697e4372b04a50d827626bb7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:21:38 UTC) from {username='kudu', principal='kudu/127.24.0.190@KRBTEST.COM'} at 127.0.0.1:40166
I20260812 01:21:38.116394 15647 tablet_service.cc:1514] Processing DeleteTablet for tablet 727a26a4697e4372b04a50d827626bb7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:21:38 UTC) from {username='kudu', principal='kudu/127.24.0.190@KRBTEST.COM'} at 127.0.0.1:45552
I20260812 01:21:38.118141 15893 tablet_service.cc:1514] Processing DeleteTablet for tablet 727a26a4697e4372b04a50d827626bb7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:21:38 UTC) from {username='kudu', principal='kudu/127.24.0.190@KRBTEST.COM'} at 127.0.0.1:57664
I20260812 01:21:38.122531 16077 tablet_replica.cc:330] T 727a26a4697e4372b04a50d827626bb7 P 0748ca23dafe4fc88570234338ffdec1: stopping tablet replica
I20260812 01:21:38.122666 16077 raft_consensus.cc:2229] T 727a26a4697e4372b04a50d827626bb7 P 0748ca23dafe4fc88570234338ffdec1 [term 1 LEADER]: Raft consensus shutting down.
I20260812 01:21:38.122794 16077 raft_consensus.cc:2258] T 727a26a4697e4372b04a50d827626bb7 P 0748ca23dafe4fc88570234338ffdec1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:21:38.124003 16077 ts_tablet_manager.cc:1905] T 727a26a4697e4372b04a50d827626bb7 P 0748ca23dafe4fc88570234338ffdec1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:21:38.124719 16078 tablet_replica.cc:330] T 727a26a4697e4372b04a50d827626bb7 P 0d48ccc189c44aa59f80d12affe2a109: stopping tablet replica
I20260812 01:21:38.124855 16078 raft_consensus.cc:2229] T 727a26a4697e4372b04a50d827626bb7 P 0d48ccc189c44aa59f80d12affe2a109 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 01:21:38.124971 16078 raft_consensus.cc:2258] T 727a26a4697e4372b04a50d827626bb7 P 0d48ccc189c44aa59f80d12affe2a109 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:21:38.125591 16078 ts_tablet_manager.cc:1905] T 727a26a4697e4372b04a50d827626bb7 P 0d48ccc189c44aa59f80d12affe2a109: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:21:38.126303 16077 ts_tablet_manager.cc:1918] T 727a26a4697e4372b04a50d827626bb7 P 0748ca23dafe4fc88570234338ffdec1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 01:21:38.126435 16077 log.cc:1196] T 727a26a4697e4372b04a50d827626bb7 P 0748ca23dafe4fc88570234338ffdec1: Deleting WAL directory at /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-1/wal/wals/727a26a4697e4372b04a50d827626bb7
I20260812 01:21:38.126778 16077 ts_tablet_manager.cc:1939] T 727a26a4697e4372b04a50d827626bb7 P 0748ca23dafe4fc88570234338ffdec1: Deleting consensus metadata
I20260812 01:21:38.127242 16078 ts_tablet_manager.cc:1918] T 727a26a4697e4372b04a50d827626bb7 P 0d48ccc189c44aa59f80d12affe2a109: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 01:21:38.127269 15183 catalog_manager.cc:4841] TS 0748ca23dafe4fc88570234338ffdec1 (127.24.0.130:35331): tablet 727a26a4697e4372b04a50d827626bb7 (table test-table [id=0c910d5f128d4a21b6b824e1275cc5ec]) successfully deleted
I20260812 01:21:38.127350 16078 log.cc:1196] T 727a26a4697e4372b04a50d827626bb7 P 0d48ccc189c44aa59f80d12affe2a109: Deleting WAL directory at /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-2/wal/wals/727a26a4697e4372b04a50d827626bb7
I20260812 01:21:38.127645 16078 ts_tablet_manager.cc:1939] T 727a26a4697e4372b04a50d827626bb7 P 0d48ccc189c44aa59f80d12affe2a109: Deleting consensus metadata
I20260812 01:21:38.128086 15183 catalog_manager.cc:4841] TS 0d48ccc189c44aa59f80d12affe2a109 (127.24.0.131:43669): tablet 727a26a4697e4372b04a50d827626bb7 (table test-table [id=0c910d5f128d4a21b6b824e1275cc5ec]) successfully deleted
W20260812 01:21:38.130098 15182 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.0.129:35121 (error 108)
I20260812 01:21:38.130628 16026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:38.130485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.129:35121 (local address 127.0.0.1:40170)
0812 01:21:38.130550 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:38.130590 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.129:35121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260812 01:21:38.130718 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 15536
W20260812 01:21:38.136215 15182 catalog_manager.cc:4584] TS 41ce5ec82b034993aaa0a6e2617407a7 (127.24.0.129:35121): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 727a26a4697e4372b04a50d827626bb7: Network error: Client connection negotiation failed: client connection to 127.24.0.129:35121: connect: Connection refused (error 111)
I20260812 01:21:38.141690 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 15768
I20260812 01:21:38.154964 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 15157
I20260812 01:21:38.168113 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.190:38787
--webserver_interface=127.24.0.190
--webserver_port=45909
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab
--principal=kudu/127.24.0.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:33767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.0.190:38787
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:21:38.252527 16081 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:21:38.252758 16081 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:21:38.252801 16081 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:21:38.253939 16081 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:21:38.254006 16081 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:21:38.254040 16081 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:21:38.254072 16081 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:21:38.254096 16081 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:21:38.255584 16081 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:33767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.24.0.190:38787
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab
--principal=kudu/127.24.0.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.190:38787
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.0.190
--webserver_port=45909
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16081
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:21:38.255862 16081 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:21:38.256116 16081 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:21:38.265678 16092 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:21:38.265703 16093 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:21:38.265676 16096 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:21:38.265961 16081 server_base.cc:1034] running on GCE node
I20260812 01:21:38.266285 16081 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:21:38.266618 16081 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:21:38.267781 16081 hybrid_clock.cc:648] HybridClock initialized: now 1786497698267770 us; error 43 us; skew 500 ppm
Aug 12 01:21:38 dist-test-slave-t3q3 krb5kdc[15138](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497698, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:21:38.270318 16081 init.cc:377] Logged in from keytab as kudu/127.24.0.190@KRBTEST.COM (short username kudu)
I20260812 01:21:38.270805 16081 webserver.cc:460] Webserver started at http://127.24.0.190:45909/ using document root <none> and password file <none>
I20260812 01:21:38.271046 16081 fs_manager.cc:359] Metadata directory not provided
I20260812 01:21:38.271130 16081 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:21:38.272603 16081 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 01:21:38.273289 16108 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:21:38.273463 16081 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 01:21:38.273571 16081 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/master-0/wal
uuid: "eea6b06caffa4bfc8b5524bfacd0c7d3"
format_stamp: "Formatted at 2026-08-12 01:21:37 on dist-test-slave-t3q3"
server_key: "f819090b3b06392cede4af5a071fe241"
server_key_iv: "36a9b49b4a3a8d53137fed665012b789"
server_key_version: "encryptionkey@0"
I20260812 01:21:38.273958 16081 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:21:38.298319 16081 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:21:38.300275 16081 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:21:38.300436 16081 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:21:38.304701 16081 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.190:38787
I20260812 01:21:38.304792 16201 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.190:38787 every 8 connection(s)
I20260812 01:21:38.305161 16081 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
I20260812 01:21:38.306849 16202 sys_catalog.cc:263] Verifying existing consensus state
I20260812 01:21:38.307542 16202 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eea6b06caffa4bfc8b5524bfacd0c7d3: Bootstrap starting.
I20260812 01:21:38.313598 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 16081
I20260812 01:21:38.313918 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.0.129:35121
--local_ip_for_outbound_sockets=127.24.0.129
--tserver_master_addrs=127.24.0.190:38787
--webserver_port=33399
--webserver_interface=127.24.0.129
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.129.keytab
--principal=kudu/127.24.0.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:33767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 01:21:38.316387 16202 log.cc:824] T 00000000000000000000000000000000 P eea6b06caffa4bfc8b5524bfacd0c7d3: Log is configured to *not* fsync() on all Append() calls
I20260812 01:21:38.318266 16202 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eea6b06caffa4bfc8b5524bfacd0c7d3: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 01:21:38.318568 16202 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eea6b06caffa4bfc8b5524bfacd0c7d3: Bootstrap complete.
I20260812 01:21:38.320961 16202 raft_consensus.cc:348] T 00000000000000000000000000000000 P eea6b06caffa4bfc8b5524bfacd0c7d3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eea6b06caffa4bfc8b5524bfacd0c7d3" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 38787 } }
I20260812 01:21:38.321249 16202 raft_consensus.cc:729] T 00000000000000000000000000000000 P eea6b06caffa4bfc8b5524bfacd0c7d3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: eea6b06caffa4bfc8b5524bfacd0c7d3, State: Initialized, Role: FOLLOWER
I20260812 01:21:38.321439 16202 consensus_queue.cc:260] T 00000000000000000000000000000000 P eea6b06caffa4bfc8b5524bfacd0c7d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eea6b06caffa4bfc8b5524bfacd0c7d3" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 38787 } }
I20260812 01:21:38.321519 16202 raft_consensus.cc:388] T 00000000000000000000000000000000 P eea6b06caffa4bfc8b5524bfacd0c7d3 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:21:38.321578 16202 raft_consensus.cc:482] T 00000000000000000000000000000000 P eea6b06caffa4bfc8b5524bfacd0c7d3 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:21:38.321646 16202 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eea6b06caffa4bfc8b5524bfacd0c7d3 [term 1 FOLLOWER]: Advancing to term 2
I20260812 01:21:38.322638 16202 raft_consensus.cc:504] T 00000000000000000000000000000000 P eea6b06caffa4bfc8b5524bfacd0c7d3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eea6b06caffa4bfc8b5524bfacd0c7d3" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 38787 } }
I20260812 01:21:38.322759 16202 leader_election.cc:302] T 00000000000000000000000000000000 P eea6b06caffa4bfc8b5524bfacd0c7d3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eea6b06caffa4bfc8b5524bfacd0c7d3; no voters:
I20260812 01:21:38.322957 16202 leader_election.cc:288] T 00000000000000000000000000000000 P eea6b06caffa4bfc8b5524bfacd0c7d3 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 01:21:38.322992 16211 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eea6b06caffa4bfc8b5524bfacd0c7d3 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 01:21:38.323099 16211 raft_consensus.cc:686] T 00000000000000000000000000000000 P eea6b06caffa4bfc8b5524bfacd0c7d3 [term 2 LEADER]: Becoming Leader. State: Replica: eea6b06caffa4bfc8b5524bfacd0c7d3, State: Running, Role: LEADER
I20260812 01:21:38.323249 16202 sys_catalog.cc:564] T 00000000000000000000000000000000 P eea6b06caffa4bfc8b5524bfacd0c7d3 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:21:38.323284 16211 consensus_queue.cc:237] T 00000000000000000000000000000000 P eea6b06caffa4bfc8b5524bfacd0c7d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eea6b06caffa4bfc8b5524bfacd0c7d3" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 38787 } }
I20260812 01:21:38.323552 16211 sys_catalog.cc:455] T 00000000000000000000000000000000 P eea6b06caffa4bfc8b5524bfacd0c7d3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eea6b06caffa4bfc8b5524bfacd0c7d3. Latest consensus state: current_term: 2 leader_uuid: "eea6b06caffa4bfc8b5524bfacd0c7d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eea6b06caffa4bfc8b5524bfacd0c7d3" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 38787 } } }
I20260812 01:21:38.323817 16211 sys_catalog.cc:458] T 00000000000000000000000000000000 P eea6b06caffa4bfc8b5524bfacd0c7d3 [sys.catalog]: This master's current role is: LEADER
I20260812 01:21:38.323729 16212 sys_catalog.cc:455] T 00000000000000000000000000000000 P eea6b06caffa4bfc8b5524bfacd0c7d3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "eea6b06caffa4bfc8b5524bfacd0c7d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eea6b06caffa4bfc8b5524bfacd0c7d3" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 38787 } } }
I20260812 01:21:38.324276 16212 sys_catalog.cc:458] T 00000000000000000000000000000000 P eea6b06caffa4bfc8b5524bfacd0c7d3 [sys.catalog]: This master's current role is: LEADER
I20260812 01:21:38.324268 16220 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:21:38.325276 16220 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:21:38.325405 16220 catalog_manager.cc:1259] Loaded cluster ID: 5434671b92c84d6294600128656e9035
I20260812 01:21:38.325448 16220 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 01:21:38.325678 16220 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260812 01:21:38.330860 16220 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260812 01:21:38.333266 16220 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:21:38.333459 16220 catalog_manager.cc:5879] T 00000000000000000000000000000000 P eea6b06caffa4bfc8b5524bfacd0c7d3: Loaded TSK: 0
I20260812 01:21:38.333683 16220 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 01:21:38.401384 16206 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:21:38.401584 16206 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:21:38.401607 16206 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:21:38.402722 16206 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:21:38.402765 16206 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:21:38.402791 16206 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.0.129
I20260812 01:21:38.404230 16206 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:33767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.129.keytab
--principal=kudu/127.24.0.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.0.129:35121
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.0.129
--webserver_port=33399
--tserver_master_addrs=127.24.0.190:38787
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16206
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.0.129
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:21:38.404457 16206 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:21:38.404700 16206 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:21:38.409776 16246 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:21:38.409884 16245 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:21:38.409750 16248 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:21:38.410019 16206 server_base.cc:1034] running on GCE node
I20260812 01:21:38.410310 16206 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:21:38.410607 16206 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:21:38.411800 16206 hybrid_clock.cc:648] HybridClock initialized: now 1786497698411788 us; error 30 us; skew 500 ppm
Aug 12 01:21:38 dist-test-slave-t3q3 krb5kdc[15138](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497698, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:21:38.418655 16206 init.cc:377] Logged in from keytab as kudu/127.24.0.129@KRBTEST.COM (short username kudu)
I20260812 01:21:38.419241 16206 webserver.cc:460] Webserver started at http://127.24.0.129:33399/ using document root <none> and password file <none>
I20260812 01:21:38.419498 16206 fs_manager.cc:359] Metadata directory not provided
I20260812 01:21:38.419586 16206 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:21:38.421367 16206 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:21:38.422147 16257 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:21:38.422334 16206 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:21:38.422439 16206 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-0/wal
uuid: "41ce5ec82b034993aaa0a6e2617407a7"
format_stamp: "Formatted at 2026-08-12 01:21:37 on dist-test-slave-t3q3"
server_key: "2703cfff634566d3398a91cf0eb6bd6c"
server_key_iv: "f00c7bc63e81c634d05b20bbe7f55b33"
server_key_version: "encryptionkey@0"
I20260812 01:21:38.422799 16206 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:21:38.447937 16206 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:21:38.450544 16206 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:21:38.450742 16206 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:21:38.451018 16206 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:21:38.451601 16271 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 01:21:38.453167 16206 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 01:21:38.453222 16206 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260812 01:21:38.453255 16206 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 01:21:38.453949 16206 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 01:21:38.453996 16206 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260812 01:21:38.454046 16271 tablet_bootstrap.cc:492] T 727a26a4697e4372b04a50d827626bb7 P 41ce5ec82b034993aaa0a6e2617407a7: Bootstrap starting.
I20260812 01:21:38.461957 16206 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.129:35121
I20260812 01:21:38.462057 16444 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.129:35121 every 8 connection(s)
I20260812 01:21:38.462532 16206 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-0/data/info.pb
I20260812 01:21:38.465675 16271 log.cc:824] T 727a26a4697e4372b04a50d827626bb7 P 41ce5ec82b034993aaa0a6e2617407a7: Log is configured to *not* fsync() on all Append() calls
I20260812 01:21:38.467799 16271 tablet_bootstrap.cc:492] T 727a26a4697e4372b04a50d827626bb7 P 41ce5ec82b034993aaa0a6e2617407a7: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 01:21:38.468191 16271 tablet_bootstrap.cc:492] T 727a26a4697e4372b04a50d827626bb7 P 41ce5ec82b034993aaa0a6e2617407a7: Bootstrap complete.
I20260812 01:21:38.469321 16271 ts_tablet_manager.cc:1397] T 727a26a4697e4372b04a50d827626bb7 P 41ce5ec82b034993aaa0a6e2617407a7: Time spent bootstrapping tablet: real 0.015s user 0.003s sys 0.010s
Aug 12 01:21:38 dist-test-slave-t3q3 krb5kdc[15138](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497698, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.129@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
I20260812 01:21:38.470261 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 16206
I20260812 01:21:38.470530 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.0.130:35331
--local_ip_for_outbound_sockets=127.24.0.130
--tserver_master_addrs=127.24.0.190:38787
--webserver_port=40319
--webserver_interface=127.24.0.130
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.130.keytab
--principal=kudu/127.24.0.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:33767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 01:21:38.470556 16271 raft_consensus.cc:348] T 727a26a4697e4372b04a50d827626bb7 P 41ce5ec82b034993aaa0a6e2617407a7 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0748ca23dafe4fc88570234338ffdec1" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 35331 } } peers { permanent_uuid: "41ce5ec82b034993aaa0a6e2617407a7" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 35121 } } peers { permanent_uuid: "0d48ccc189c44aa59f80d12affe2a109" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 43669 } }
I20260812 01:21:38.470966 16271 raft_consensus.cc:729] T 727a26a4697e4372b04a50d827626bb7 P 41ce5ec82b034993aaa0a6e2617407a7 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41ce5ec82b034993aaa0a6e2617407a7, State: Initialized, Role: FOLLOWER
I20260812 01:21:38.471136 16271 consensus_queue.cc:260] T 727a26a4697e4372b04a50d827626bb7 P 41ce5ec82b034993aaa0a6e2617407a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0748ca23dafe4fc88570234338ffdec1" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 35331 } } peers { permanent_uuid: "41ce5ec82b034993aaa0a6e2617407a7" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 35121 } } peers { permanent_uuid: "0d48ccc189c44aa59f80d12affe2a109" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 43669 } }
I20260812 01:21:38.471426 16271 ts_tablet_manager.cc:1428] T 727a26a4697e4372b04a50d827626bb7 P 41ce5ec82b034993aaa0a6e2617407a7: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260812 01:21:38.474386 16447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:38.463293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:38787 (local address 127.24.0.129:54209)
0812 01:21:38.463500 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:38.463543 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 01:21:38.464047 (+ 504us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:21:38.464545 (+ 498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:21:38.464549 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:21:38.464845 (+ 296us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:21:38.465041 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:21:38.465046 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:21:38.466404 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:21:38.466405 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:21:38.467219 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:21:38.467221 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:21:38.467849 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:21:38.467851 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:21:38.467991 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:38.468620 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:21:38.468629 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:21:38.470265 (+ 1636us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:21:38.472704 (+ 2439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:21:38.472711 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:21:38.472714 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:21:38.473058 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:21:38.473323 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:21:38.473324 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:21:38.473325 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:21:38.473467 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:21:38.473838 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:21:38.473845 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:21:38.474007 (+ 162us) client_negotiation.cc:769] Sending connection context
0812 01:21:38.474118 (+ 111us) client_negotiation.cc:240] Negotiation successful
0812 01:21:38.474246 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":122,"threads_started":1}
I20260812 01:21:38.474606 16448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:38.463427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.129:54209 (local address 127.24.0.190:38787)
0812 01:21:38.463636 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0812 01:21:38.463644 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 01:21:38.463687 (+ 43us) server_negotiation.cc:407] Connection header received
0812 01:21:38.464326 (+ 639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:21:38.464332 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:21:38.464375 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:21:38.464513 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:21:38.465166 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:38.466284 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:38.467329 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:38.467730 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:38.467845 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:38.470427 (+ 2582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:21:38.470432 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:21:38.470432 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:21:38.470446 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:21:38.472599 (+ 2153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:21:38.473158 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:21:38.473160 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:21:38.473161 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:21:38.473251 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:21:38.473537 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:21:38.473537 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:21:38.473537 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:21:38.473765 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:21:38.473832 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 01:21:38.474214 (+ 382us) server_negotiation.cc:299] Negotiation successful
0812 01:21:38.474373 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":102,"threads_started":1}
I20260812 01:21:38.475217 16445 heartbeater.cc:347] Connected to a master server at 127.24.0.190:38787
I20260812 01:21:38.475313 16445 heartbeater.cc:464] Registering TS with master...
I20260812 01:21:38.475481 16445 heartbeater.cc:511] Master 127.24.0.190:38787 requested a full tablet report, sending...
I20260812 01:21:38.476245 16137 ts_manager.cc:194] Registered new tserver with Master: 41ce5ec82b034993aaa0a6e2617407a7 (127.24.0.129:35121)
I20260812 01:21:38.477447 16137 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.0.129@KRBTEST.COM'} at 127.24.0.129:54209
I20260812 01:21:38.478406 16445 heartbeater.cc:503] Master 127.24.0.190:38787 was elected leader, sending a full tablet report...
Aug 12 01:21:38 dist-test-slave-t3q3 krb5kdc[15138](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497698, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.190@KRBTEST.COM for kudu/127.24.0.129@KRBTEST.COM
I20260812 01:21:38.485765 16456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:38.477079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40176 (local address 127.24.0.129:35121)
0812 01:21:38.477252 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0812 01:21:38.477256 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 01:21:38.477266 (+ 10us) server_negotiation.cc:407] Connection header received
0812 01:21:38.477297 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:21:38.477299 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:21:38.477314 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:21:38.477391 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:21:38.478033 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:38.479034 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:38.480248 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:38.480523 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:38.480584 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:38.482849 (+ 2265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:21:38.482858 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:21:38.482858 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:21:38.482866 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:21:38.484478 (+ 1612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:21:38.484897 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:21:38.484900 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:21:38.484901 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:21:38.484962 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:21:38.485167 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:21:38.485168 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:21:38.485168 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:21:38.485307 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:21:38.485395 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0812 01:21:38.485583 (+ 188us) server_negotiation.cc:299] Negotiation successful
0812 01:21:38.485671 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":148,"thread_start_us":84,"threads_started":1}
I20260812 01:21:38.485765 16455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:38.476856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.129:35121 (local address 127.0.0.1:40176)
0812 01:21:38.477046 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:38.477088 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 01:21:38.477168 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:21:38.477409 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:21:38.477416 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:21:38.477869 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:21:38.477942 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:21:38.477946 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:21:38.479111 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:21:38.479112 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:21:38.480169 (+ 1057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:21:38.480172 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:21:38.480613 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:21:38.480615 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:21:38.480712 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:38.481199 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:21:38.481206 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:21:38.482758 (+ 1552us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:21:38.484584 (+ 1826us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:21:38.484591 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:21:38.484593 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:21:38.484795 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:21:38.485036 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:21:38.485037 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:21:38.485037 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:21:38.485105 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:21:38.485386 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:21:38.485392 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:21:38.485445 (+ 53us) client_negotiation.cc:769] Sending connection context
0812 01:21:38.485544 (+ 99us) client_negotiation.cc:240] Negotiation successful
0812 01:21:38.485672 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":98,"threads_started":1}
I20260812 01:21:38.486128 16338 tablet_service.cc:1514] Processing DeleteTablet for tablet 727a26a4697e4372b04a50d827626bb7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:21:38 UTC) from {username='kudu', principal='kudu/127.24.0.190@KRBTEST.COM'} at 127.0.0.1:40176
I20260812 01:21:38.486637 16461 tablet_replica.cc:330] stopping tablet replica
I20260812 01:21:38.486753 16461 raft_consensus.cc:2229] T 727a26a4697e4372b04a50d827626bb7 P 41ce5ec82b034993aaa0a6e2617407a7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 01:21:38.486821 16461 raft_consensus.cc:2258] T 727a26a4697e4372b04a50d827626bb7 P 41ce5ec82b034993aaa0a6e2617407a7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:21:38.487131 16461 ts_tablet_manager.cc:1905] T 727a26a4697e4372b04a50d827626bb7 P 41ce5ec82b034993aaa0a6e2617407a7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:21:38.489317 16461 ts_tablet_manager.cc:1918] T 727a26a4697e4372b04a50d827626bb7 P 41ce5ec82b034993aaa0a6e2617407a7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 01:21:38.489396 16461 log.cc:1196] T 727a26a4697e4372b04a50d827626bb7 P 41ce5ec82b034993aaa0a6e2617407a7: Deleting WAL directory at /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-0/wal/wals/727a26a4697e4372b04a50d827626bb7
I20260812 01:21:38.489650 16461 ts_tablet_manager.cc:1939] T 727a26a4697e4372b04a50d827626bb7 P 41ce5ec82b034993aaa0a6e2617407a7: Deleting consensus metadata
I20260812 01:21:38.490162 16109 catalog_manager.cc:4841] TS 41ce5ec82b034993aaa0a6e2617407a7 (127.24.0.129:35121): tablet 727a26a4697e4372b04a50d827626bb7 (table test-table [id=0c910d5f128d4a21b6b824e1275cc5ec]) successfully deleted
W20260812 01:21:38.560590 16454 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:21:38.560789 16454 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:21:38.560813 16454 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:21:38.561987 16454 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:21:38.562033 16454 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:21:38.562060 16454 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.0.130
I20260812 01:21:38.563495 16454 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:33767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.130.keytab
--principal=kudu/127.24.0.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.0.130:35331
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.0.130
--webserver_port=40319
--tserver_master_addrs=127.24.0.190:38787
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16454
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.0.130
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:21:38.563747 16454 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:21:38.563980 16454 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:21:38.568902 16471 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:21:38.568930 16469 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:21:38.568912 16468 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:21:38.569248 16454 server_base.cc:1034] running on GCE node
I20260812 01:21:38.569461 16454 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:21:38.569731 16454 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:21:38.570897 16454 hybrid_clock.cc:648] HybridClock initialized: now 1786497698570892 us; error 37 us; skew 500 ppm
Aug 12 01:21:38 dist-test-slave-t3q3 krb5kdc[15138](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497698, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:21:38.573386 16454 init.cc:377] Logged in from keytab as kudu/127.24.0.130@KRBTEST.COM (short username kudu)
I20260812 01:21:38.573860 16454 webserver.cc:460] Webserver started at http://127.24.0.130:40319/ using document root <none> and password file <none>
I20260812 01:21:38.574088 16454 fs_manager.cc:359] Metadata directory not provided
I20260812 01:21:38.574173 16454 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:21:38.575500 16454 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:21:38.576112 16487 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:21:38.576305 16454 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:21:38.576404 16454 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-1/wal
uuid: "0748ca23dafe4fc88570234338ffdec1"
format_stamp: "Formatted at 2026-08-12 01:21:37 on dist-test-slave-t3q3"
server_key: "5f044f54d62149cb82904befdb3ad5cb"
server_key_iv: "5fd9fa8d22942fcd892eee45cda85e51"
server_key_version: "encryptionkey@0"
I20260812 01:21:38.576756 16454 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:21:38.589323 16454 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:21:38.591089 16454 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:21:38.591252 16454 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:21:38.591475 16454 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:21:38.591787 16454 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:21:38.591843 16454 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:21:38.591883 16454 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:21:38.591912 16454 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:21:38.597987 16454 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.130:35331
I20260812 01:21:38.598040 16664 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.130:35331 every 8 connection(s)
I20260812 01:21:38.598384 16454 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-1/data/info.pb
Aug 12 01:21:38 dist-test-slave-t3q3 krb5kdc[15138](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497698, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.130@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
I20260812 01:21:38.607482 16448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:38.599119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.130:34681 (local address 127.24.0.190:38787)
0812 01:21:38.599226 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0812 01:21:38.599229 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:21:38.599407 (+ 178us) server_negotiation.cc:407] Connection header received
0812 01:21:38.599875 (+ 468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:21:38.599876 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:21:38.599886 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:21:38.599938 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:21:38.600881 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:38.601383 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:38.602332 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:38.602525 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:38.602576 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:38.604880 (+ 2304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:21:38.604882 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:21:38.604882 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:21:38.604889 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:21:38.606346 (+ 1457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:21:38.606726 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:21:38.606727 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:21:38.606727 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:21:38.606779 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:21:38.606955 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:21:38.606956 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:21:38.606956 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:21:38.607057 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:21:38.607107 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 01:21:38.607346 (+ 239us) server_negotiation.cc:299] Negotiation successful
0812 01:21:38.607416 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 01:21:38.607756 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 16454
I20260812 01:21:38.608001 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.0.131:43669
--local_ip_for_outbound_sockets=127.24.0.131
--tserver_master_addrs=127.24.0.190:38787
--webserver_port=42015
--webserver_interface=127.24.0.131
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.131.keytab
--principal=kudu/127.24.0.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:33767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 01:21:38.608265 16667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:38.599094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:38787 (local address 127.24.0.130:34681)
0812 01:21:38.599288 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:38.599324 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 01:21:38.599795 (+ 471us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:21:38.599992 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:21:38.599995 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:21:38.600607 (+ 612us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:21:38.600800 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:21:38.600804 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:21:38.601466 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:21:38.601467 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:21:38.602260 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:21:38.602261 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:21:38.602597 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:21:38.602598 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:21:38.602695 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:38.603249 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:21:38.603264 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:21:38.604789 (+ 1525us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:21:38.606419 (+ 1630us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:21:38.606425 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:21:38.606427 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:21:38.606658 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:21:38.606823 (+ 165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:21:38.606824 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:21:38.606824 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:21:38.606908 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:21:38.607113 (+ 205us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:21:38.607118 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:21:38.607235 (+ 117us) client_negotiation.cc:769] Sending connection context
0812 01:21:38.607319 (+ 84us) client_negotiation.cc:240] Negotiation successful
0812 01:21:38.607437 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":117,"threads_started":1}
I20260812 01:21:38.608884 16665 heartbeater.cc:347] Connected to a master server at 127.24.0.190:38787
I20260812 01:21:38.608994 16665 heartbeater.cc:464] Registering TS with master...
I20260812 01:21:38.609213 16665 heartbeater.cc:511] Master 127.24.0.190:38787 requested a full tablet report, sending...
I20260812 01:21:38.609608 16137 ts_manager.cc:194] Registered new tserver with Master: 0748ca23dafe4fc88570234338ffdec1 (127.24.0.130:35331)
I20260812 01:21:38.609987 16137 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.0.130@KRBTEST.COM'} at 127.24.0.130:34681
W20260812 01:21:38.692016 16669 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:21:38.692278 16669 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:21:38.692302 16669 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:21:38.693387 16669 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:21:38.693423 16669 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:21:38.693447 16669 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.0.131
I20260812 01:21:38.694876 16669 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:33767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.131.keytab
--principal=kudu/127.24.0.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.0.131:43669
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.0.131
--webserver_port=42015
--tserver_master_addrs=127.24.0.190:38787
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16669
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.0.131
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:21:38.695058 16669 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:21:38.695236 16669 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:21:38.700217 16683 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:21:38.700204 16681 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:21:38.700217 16680 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:21:38.700495 16669 server_base.cc:1034] running on GCE node
I20260812 01:21:38.700798 16669 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:21:38.701049 16669 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:21:38.702224 16669 hybrid_clock.cc:648] HybridClock initialized: now 1786497698702200 us; error 32 us; skew 500 ppm
Aug 12 01:21:38 dist-test-slave-t3q3 krb5kdc[15138](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497698, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:21:38.704720 16669 init.cc:377] Logged in from keytab as kudu/127.24.0.131@KRBTEST.COM (short username kudu)
I20260812 01:21:38.705193 16669 webserver.cc:460] Webserver started at http://127.24.0.131:42015/ using document root <none> and password file <none>
I20260812 01:21:38.705420 16669 fs_manager.cc:359] Metadata directory not provided
I20260812 01:21:38.705500 16669 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:21:38.706835 16669 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:21:38.707396 16693 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:21:38.707566 16669 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 01:21:38.707676 16669 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-2/wal
uuid: "0d48ccc189c44aa59f80d12affe2a109"
format_stamp: "Formatted at 2026-08-12 01:21:37 on dist-test-slave-t3q3"
server_key: "4e8e79c992c5758da2ada0484c88e2bb"
server_key_iv: "b1bd463be88d537e77a43a1b4ef41879"
server_key_version: "encryptionkey@0"
I20260812 01:21:38.707998 16669 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:21:38.716878 16669 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:21:38.718650 16669 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:21:38.718756 16669 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:21:38.718928 16669 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:21:38.719208 16669 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:21:38.719244 16669 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:21:38.719264 16669 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:21:38.719277 16669 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:21:38.725497 16669 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.131:43669
I20260812 01:21:38.725533 16868 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.131:43669 every 8 connection(s)
I20260812 01:21:38.725929 16669 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-2/data/info.pb
Aug 12 01:21:38 dist-test-slave-t3q3 krb5kdc[15138](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497698, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.131@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
I20260812 01:21:38.734411 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 16669
I20260812 01:21:38.736521 16448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:38.726605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.131:56163 (local address 127.24.0.190:38787)
0812 01:21:38.726687 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0812 01:21:38.726689 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:21:38.726922 (+ 233us) server_negotiation.cc:407] Connection header received
0812 01:21:38.727432 (+ 510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:21:38.727433 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:21:38.727446 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:21:38.727494 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:21:38.728584 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:38.729072 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:38.730045 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:38.730250 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:38.730329 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:38.732660 (+ 2331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:21:38.732662 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:21:38.732662 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:21:38.732669 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:21:38.734424 (+ 1755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:21:38.734919 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:21:38.734920 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:21:38.734921 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:21:38.734999 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:21:38.735722 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:21:38.735723 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:21:38.735724 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:21:38.735876 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:21:38.735928 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 01:21:38.736348 (+ 420us) server_negotiation.cc:299] Negotiation successful
0812 01:21:38.736432 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 01:21:38.736722 16875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:38.726582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:38787 (local address 127.24.0.131:56163)
0812 01:21:38.726780 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:38.726819 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 01:21:38.727324 (+ 505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:21:38.727575 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:21:38.727579 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:21:38.728275 (+ 696us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:21:38.728486 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:21:38.728490 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:21:38.729168 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:21:38.729169 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:21:38.729948 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:21:38.729950 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:21:38.730336 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:21:38.730337 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:21:38.730465 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:38.731082 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:21:38.731094 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:21:38.732538 (+ 1444us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:21:38.734540 (+ 2002us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:21:38.734548 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:21:38.734550 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:21:38.734830 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:21:38.735447 (+ 617us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:21:38.735448 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:21:38.735449 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:21:38.735619 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:21:38.735983 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:21:38.735990 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:21:38.736175 (+ 185us) client_negotiation.cc:769] Sending connection context
0812 01:21:38.736317 (+ 142us) client_negotiation.cc:240] Negotiation successful
0812 01:21:38.736473 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":116,"threads_started":1}
I20260812 01:21:38.737300 16870 heartbeater.cc:347] Connected to a master server at 127.24.0.190:38787
I20260812 01:21:38.737413 16870 heartbeater.cc:464] Registering TS with master...
I20260812 01:21:38.737600 16870 heartbeater.cc:511] Master 127.24.0.190:38787 requested a full tablet report, sending...
I20260812 01:21:38.738015 16137 ts_manager.cc:194] Registered new tserver with Master: 0d48ccc189c44aa59f80d12affe2a109 (127.24.0.131:43669)
I20260812 01:21:38.738371 16137 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.0.131@KRBTEST.COM'} at 127.24.0.131:56163
I20260812 01:21:38.742012 16881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:38.735180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54836 (local address 127.24.0.190:38787)
0812 01:21:38.735300 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0812 01:21:38.735302 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:21:38.735311 (+ 9us) server_negotiation.cc:407] Connection header received
0812 01:21:38.735332 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:21:38.735333 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:21:38.735351 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:21:38.735443 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:21:38.736609 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:38.737305 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:38.738351 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:38.738625 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:38.738698 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:38.739866 (+ 1168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:21:38.739867 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:21:38.739867 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:21:38.739873 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:21:38.741098 (+ 1225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:21:38.741408 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:21:38.741409 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:21:38.741409 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:21:38.741460 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:21:38.741635 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:21:38.741636 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:21:38.741636 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:21:38.741724 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:21:38.741840 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0812 01:21:38.741892 (+ 52us) server_negotiation.cc:299] Negotiation successful
0812 01:21:38.741950 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":81,"threads_started":1}
I20260812 01:21:38.742725 24578 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 01:21:38.745232 16881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:38.743079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54844 (local address 127.24.0.190:38787)
0812 01:21:38.743128 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 01:21:38.743130 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:21:38.743221 (+ 91us) server_negotiation.cc:407] Connection header received
0812 01:21:38.743269 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:21:38.743269 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:21:38.743294 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:21:38.743341 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:21:38.743415 (+ 74us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:38.743854 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:38.744635 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:38.744812 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:38.744866 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:38.744996 (+ 130us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:21:38.745079 (+ 83us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:21:38.745141 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 01:21:38.745172 (+ 31us) server_negotiation.cc:299] Negotiation successful
0812 01:21:38.745190 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260812 01:21:38.745502 16137 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:54844:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 01:21:38.747087 16137 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 01:21:38.749375 16338 tablet_service.cc:1467] Processing CreateTablet for tablet 4f224233417d42b989c83b7df00df3f6 (DEFAULT_TABLE table=test-table [id=0cd6e8c79b8e424d9fd195df42ab565b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:21:38.749598 16338 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f224233417d42b989c83b7df00df3f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:21:38.751104 16271 tablet_bootstrap.cc:492] T 4f224233417d42b989c83b7df00df3f6 P 41ce5ec82b034993aaa0a6e2617407a7: Bootstrap starting.
I20260812 01:21:38.751441 16271 tablet_bootstrap.cc:654] T 4f224233417d42b989c83b7df00df3f6 P 41ce5ec82b034993aaa0a6e2617407a7: Neither blocks nor log segments found. Creating new log.
I20260812 01:21:38.752365 16271 tablet_bootstrap.cc:492] T 4f224233417d42b989c83b7df00df3f6 P 41ce5ec82b034993aaa0a6e2617407a7: No bootstrap required, opened a new log
I20260812 01:21:38.752415 16271 ts_tablet_manager.cc:1397] T 4f224233417d42b989c83b7df00df3f6 P 41ce5ec82b034993aaa0a6e2617407a7: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260812 01:21:38.752557 16271 raft_consensus.cc:348] T 4f224233417d42b989c83b7df00df3f6 P 41ce5ec82b034993aaa0a6e2617407a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d48ccc189c44aa59f80d12affe2a109" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 43669 } } peers { permanent_uuid: "0748ca23dafe4fc88570234338ffdec1" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 35331 } } peers { permanent_uuid: "41ce5ec82b034993aaa0a6e2617407a7" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 35121 } }
I20260812 01:21:38.752633 16271 raft_consensus.cc:374] T 4f224233417d42b989c83b7df00df3f6 P 41ce5ec82b034993aaa0a6e2617407a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:21:38.752681 16271 raft_consensus.cc:729] T 4f224233417d42b989c83b7df00df3f6 P 41ce5ec82b034993aaa0a6e2617407a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41ce5ec82b034993aaa0a6e2617407a7, State: Initialized, Role: FOLLOWER
I20260812 01:21:38.752730 16271 consensus_queue.cc:260] T 4f224233417d42b989c83b7df00df3f6 P 41ce5ec82b034993aaa0a6e2617407a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d48ccc189c44aa59f80d12affe2a109" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 43669 } } peers { permanent_uuid: "0748ca23dafe4fc88570234338ffdec1" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 35331 } } peers { permanent_uuid: "41ce5ec82b034993aaa0a6e2617407a7" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 35121 } }
I20260812 01:21:38.752830 16271 ts_tablet_manager.cc:1428] T 4f224233417d42b989c83b7df00df3f6 P 41ce5ec82b034993aaa0a6e2617407a7: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 01:21:38.753021 16883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:38.749066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57676 (local address 127.24.0.131:43669)
0812 01:21:38.749275 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0812 01:21:38.749279 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 01:21:38.749296 (+ 17us) server_negotiation.cc:407] Connection header received
0812 01:21:38.749327 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:21:38.749329 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:21:38.749341 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:21:38.749408 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:21:38.749552 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:38.750564 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:38.751791 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:38.752574 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:38.752655 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:38.752730 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 01:21:38.752885 (+ 155us) server_negotiation.cc:299] Negotiation successful
0812 01:21:38.752925 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":179,"thread_start_us":106,"threads_started":1}
I20260812 01:21:38.753919 16455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:38.748989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.131:43669 (local address 127.0.0.1:57676)
0812 01:21:38.749134 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:38.749147 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 01:21:38.749190 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:21:38.749419 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:21:38.749419 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:21:38.749421 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:21:38.749465 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:21:38.749466 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:21:38.750689 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:21:38.750689 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:21:38.751653 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:21:38.751657 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:21:38.752666 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:21:38.752667 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:21:38.752749 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:38.752773 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 01:21:38.752826 (+ 53us) client_negotiation.cc:240] Negotiation successful
0812 01:21:38.753871 (+ 1045us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"mutex_wait_us":1002}
I20260812 01:21:38.754623 16763 tablet_service.cc:1467] Processing CreateTablet for tablet 4f224233417d42b989c83b7df00df3f6 (DEFAULT_TABLE table=test-table [id=0cd6e8c79b8e424d9fd195df42ab565b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:21:38.754935 16884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:38.749112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.130:35331 (local address 127.0.0.1:45576)
0812 01:21:38.749284 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:38.749292 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 01:21:38.749325 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:21:38.751513 (+ 2188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:21:38.751514 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:21:38.751516 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:21:38.751558 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:21:38.751560 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:21:38.752751 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:21:38.752752 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:21:38.753803 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:21:38.753806 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:21:38.754698 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:21:38.754699 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:21:38.754774 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:38.754778 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 01:21:38.754832 (+ 54us) client_negotiation.cc:240] Negotiation successful
0812 01:21:38.754872 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":75,"threads_started":1}
I20260812 01:21:38.754962 16763 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f224233417d42b989c83b7df00df3f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:21:38.754994 16885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:38.751165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45576 (local address 127.24.0.130:35331)
0812 01:21:38.751348 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0812 01:21:38.751352 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 01:21:38.751362 (+ 10us) server_negotiation.cc:407] Connection header received
0812 01:21:38.751400 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:21:38.751407 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:21:38.751419 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:21:38.751515 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:21:38.751634 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:38.752613 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:38.754020 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:38.754599 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:38.754696 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:38.754753 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 01:21:38.754871 (+ 118us) server_negotiation.cc:299] Negotiation successful
0812 01:21:38.754909 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":109,"threads_started":1}
I20260812 01:21:38.756796 16559 tablet_service.cc:1467] Processing CreateTablet for tablet 4f224233417d42b989c83b7df00df3f6 (DEFAULT_TABLE table=test-table [id=0cd6e8c79b8e424d9fd195df42ab565b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:21:38.757164 16559 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f224233417d42b989c83b7df00df3f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:21:38.758329 16889 tablet_bootstrap.cc:492] T 4f224233417d42b989c83b7df00df3f6 P 0d48ccc189c44aa59f80d12affe2a109: Bootstrap starting.
I20260812 01:21:38.759003 16889 tablet_bootstrap.cc:654] T 4f224233417d42b989c83b7df00df3f6 P 0d48ccc189c44aa59f80d12affe2a109: Neither blocks nor log segments found. Creating new log.
I20260812 01:21:38.759557 16889 log.cc:824] T 4f224233417d42b989c83b7df00df3f6 P 0d48ccc189c44aa59f80d12affe2a109: Log is configured to *not* fsync() on all Append() calls
I20260812 01:21:38.760383 16889 tablet_bootstrap.cc:492] T 4f224233417d42b989c83b7df00df3f6 P 0d48ccc189c44aa59f80d12affe2a109: No bootstrap required, opened a new log
I20260812 01:21:38.760460 16889 ts_tablet_manager.cc:1397] T 4f224233417d42b989c83b7df00df3f6 P 0d48ccc189c44aa59f80d12affe2a109: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 01:21:38.760789 16893 tablet_bootstrap.cc:492] T 4f224233417d42b989c83b7df00df3f6 P 0748ca23dafe4fc88570234338ffdec1: Bootstrap starting.
I20260812 01:21:38.761373 16893 tablet_bootstrap.cc:654] T 4f224233417d42b989c83b7df00df3f6 P 0748ca23dafe4fc88570234338ffdec1: Neither blocks nor log segments found. Creating new log.
I20260812 01:21:38.761991 16893 log.cc:824] T 4f224233417d42b989c83b7df00df3f6 P 0748ca23dafe4fc88570234338ffdec1: Log is configured to *not* fsync() on all Append() calls
I20260812 01:21:38.762239 16889 raft_consensus.cc:348] T 4f224233417d42b989c83b7df00df3f6 P 0d48ccc189c44aa59f80d12affe2a109 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d48ccc189c44aa59f80d12affe2a109" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 43669 } } peers { permanent_uuid: "0748ca23dafe4fc88570234338ffdec1" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 35331 } } peers { permanent_uuid: "41ce5ec82b034993aaa0a6e2617407a7" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 35121 } }
I20260812 01:21:38.762419 16889 raft_consensus.cc:374] T 4f224233417d42b989c83b7df00df3f6 P 0d48ccc189c44aa59f80d12affe2a109 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:21:38.762461 16889 raft_consensus.cc:729] T 4f224233417d42b989c83b7df00df3f6 P 0d48ccc189c44aa59f80d12affe2a109 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d48ccc189c44aa59f80d12affe2a109, State: Initialized, Role: FOLLOWER
I20260812 01:21:38.762590 16889 consensus_queue.cc:260] T 4f224233417d42b989c83b7df00df3f6 P 0d48ccc189c44aa59f80d12affe2a109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d48ccc189c44aa59f80d12affe2a109" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 43669 } } peers { permanent_uuid: "0748ca23dafe4fc88570234338ffdec1" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 35331 } } peers { permanent_uuid: "41ce5ec82b034993aaa0a6e2617407a7" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 35121 } }
I20260812 01:21:38.762820 16893 tablet_bootstrap.cc:492] T 4f224233417d42b989c83b7df00df3f6 P 0748ca23dafe4fc88570234338ffdec1: No bootstrap required, opened a new log
I20260812 01:21:38.762899 16893 ts_tablet_manager.cc:1397] T 4f224233417d42b989c83b7df00df3f6 P 0748ca23dafe4fc88570234338ffdec1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 01:21:38.762892 16870 heartbeater.cc:503] Master 127.24.0.190:38787 was elected leader, sending a full tablet report...
I20260812 01:21:38.762892 16889 ts_tablet_manager.cc:1428] T 4f224233417d42b989c83b7df00df3f6 P 0d48ccc189c44aa59f80d12affe2a109: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260812 01:21:38.764593 16893 raft_consensus.cc:348] T 4f224233417d42b989c83b7df00df3f6 P 0748ca23dafe4fc88570234338ffdec1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d48ccc189c44aa59f80d12affe2a109" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 43669 } } peers { permanent_uuid: "0748ca23dafe4fc88570234338ffdec1" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 35331 } } peers { permanent_uuid: "41ce5ec82b034993aaa0a6e2617407a7" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 35121 } }
I20260812 01:21:38.764719 16893 raft_consensus.cc:374] T 4f224233417d42b989c83b7df00df3f6 P 0748ca23dafe4fc88570234338ffdec1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:21:38.764740 16893 raft_consensus.cc:729] T 4f224233417d42b989c83b7df00df3f6 P 0748ca23dafe4fc88570234338ffdec1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0748ca23dafe4fc88570234338ffdec1, State: Initialized, Role: FOLLOWER
I20260812 01:21:38.764876 16893 consensus_queue.cc:260] T 4f224233417d42b989c83b7df00df3f6 P 0748ca23dafe4fc88570234338ffdec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d48ccc189c44aa59f80d12affe2a109" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 43669 } } peers { permanent_uuid: "0748ca23dafe4fc88570234338ffdec1" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 35331 } } peers { permanent_uuid: "41ce5ec82b034993aaa0a6e2617407a7" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 35121 } }
I20260812 01:21:38.765158 16893 ts_tablet_manager.cc:1428] T 4f224233417d42b989c83b7df00df3f6 P 0748ca23dafe4fc88570234338ffdec1: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 01:21:38.765196 16665 heartbeater.cc:503] Master 127.24.0.190:38787 was elected leader, sending a full tablet report...
W20260812 01:21:38.849013 16666 tablet.cc:2369] T 4f224233417d42b989c83b7df00df3f6 P 0748ca23dafe4fc88570234338ffdec1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 01:21:38.963481 16446 tablet.cc:2369] T 4f224233417d42b989c83b7df00df3f6 P 41ce5ec82b034993aaa0a6e2617407a7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 01:21:38.976497 16872 tablet.cc:2369] T 4f224233417d42b989c83b7df00df3f6 P 0d48ccc189c44aa59f80d12affe2a109: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 01:21:38.986356 16896 raft_consensus.cc:482] T 4f224233417d42b989c83b7df00df3f6 P 0d48ccc189c44aa59f80d12affe2a109 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 01:21:38.986466 16896 raft_consensus.cc:504] T 4f224233417d42b989c83b7df00df3f6 P 0d48ccc189c44aa59f80d12affe2a109 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d48ccc189c44aa59f80d12affe2a109" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 43669 } } peers { permanent_uuid: "0748ca23dafe4fc88570234338ffdec1" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 35331 } } peers { permanent_uuid: "41ce5ec82b034993aaa0a6e2617407a7" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 35121 } }
I20260812 01:21:38.986848 16896 leader_election.cc:288] T 4f224233417d42b989c83b7df00df3f6 P 0d48ccc189c44aa59f80d12affe2a109 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0748ca23dafe4fc88570234338ffdec1 (127.24.0.130:35331), 41ce5ec82b034993aaa0a6e2617407a7 (127.24.0.129:35121)
I20260812 01:21:38.989828 16885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:38.987130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.131:49555 (local address 127.24.0.130:35331)
0812 01:21:38.987187 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0812 01:21:38.987190 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:21:38.987234 (+ 44us) server_negotiation.cc:407] Connection header received
0812 01:21:38.987264 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:21:38.987265 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:21:38.987275 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:21:38.987334 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:21:38.987537 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:38.988076 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:38.989017 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:38.989445 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:38.989503 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:38.989510 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0812 01:21:38.989704 (+ 194us) server_negotiation.cc:299] Negotiation successful
0812 01:21:38.989746 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 01:21:38.989820 16875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:38.987058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.130:35331 (local address 127.24.0.131:49555)
0812 01:21:38.987117 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:38.987130 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 01:21:38.987177 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:21:38.987360 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:21:38.987361 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:21:38.987364 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:21:38.987447 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:21:38.987449 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:21:38.988143 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:21:38.988144 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:21:38.988921 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:21:38.988923 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:21:38.989526 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:21:38.989527 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:21:38.989611 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:38.989616 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 01:21:38.989684 (+ 68us) client_negotiation.cc:240] Negotiation successful
0812 01:21:38.989728 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260812 01:21:38.989945 16899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:38.987063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.129:35121 (local address 127.24.0.131:39683)
0812 01:21:38.987291 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:38.987299 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 01:21:38.987351 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:21:38.987537 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:21:38.987538 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:21:38.987540 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:21:38.987605 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:21:38.987607 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:21:38.988301 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:21:38.988302 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:21:38.989053 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:21:38.989055 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:21:38.989643 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:21:38.989644 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:21:38.989737 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:38.989741 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 01:21:38.989802 (+ 61us) client_negotiation.cc:240] Negotiation successful
0812 01:21:38.989836 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"mutex_wait_us":91,"thread_start_us":83,"threads_started":1}
I20260812 01:21:38.989971 16898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:38.987130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.131:39683 (local address 127.24.0.129:35121)
0812 01:21:38.987268 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0812 01:21:38.987270 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:21:38.987361 (+ 91us) server_negotiation.cc:407] Connection header received
0812 01:21:38.987396 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:21:38.987397 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:21:38.987429 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:21:38.987518 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:21:38.987674 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:38.988221 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:38.989145 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:38.989560 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:38.989633 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:38.989709 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 01:21:38.989817 (+ 108us) server_negotiation.cc:299] Negotiation successful
0812 01:21:38.989868 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":70,"threads_started":1}
I20260812 01:21:38.990221 16590 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f224233417d42b989c83b7df00df3f6" candidate_uuid: "0d48ccc189c44aa59f80d12affe2a109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0748ca23dafe4fc88570234338ffdec1" is_pre_election: true
I20260812 01:21:38.990217 16381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f224233417d42b989c83b7df00df3f6" candidate_uuid: "0d48ccc189c44aa59f80d12affe2a109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41ce5ec82b034993aaa0a6e2617407a7" is_pre_election: true
I20260812 01:21:38.990362 16590 raft_consensus.cc:2454] T 4f224233417d42b989c83b7df00df3f6 P 0748ca23dafe4fc88570234338ffdec1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d48ccc189c44aa59f80d12affe2a109 in term 0.
I20260812 01:21:38.990362 16381 raft_consensus.cc:2454] T 4f224233417d42b989c83b7df00df3f6 P 41ce5ec82b034993aaa0a6e2617407a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d48ccc189c44aa59f80d12affe2a109 in term 0.
I20260812 01:21:38.990605 16694 leader_election.cc:302] T 4f224233417d42b989c83b7df00df3f6 P 0d48ccc189c44aa59f80d12affe2a109 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d48ccc189c44aa59f80d12affe2a109, 41ce5ec82b034993aaa0a6e2617407a7; no voters:
I20260812 01:21:38.990723 16896 raft_consensus.cc:2781] T 4f224233417d42b989c83b7df00df3f6 P 0d48ccc189c44aa59f80d12affe2a109 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 01:21:38.990782 16896 raft_consensus.cc:482] T 4f224233417d42b989c83b7df00df3f6 P 0d48ccc189c44aa59f80d12affe2a109 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 01:21:38.990806 16896 raft_consensus.cc:3037] T 4f224233417d42b989c83b7df00df3f6 P 0d48ccc189c44aa59f80d12affe2a109 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:21:38.991514 16896 raft_consensus.cc:504] T 4f224233417d42b989c83b7df00df3f6 P 0d48ccc189c44aa59f80d12affe2a109 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d48ccc189c44aa59f80d12affe2a109" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 43669 } } peers { permanent_uuid: "0748ca23dafe4fc88570234338ffdec1" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 35331 } } peers { permanent_uuid: "41ce5ec82b034993aaa0a6e2617407a7" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 35121 } }
I20260812 01:21:38.991669 16896 leader_election.cc:288] T 4f224233417d42b989c83b7df00df3f6 P 0d48ccc189c44aa59f80d12affe2a109 [CANDIDATE]: Term 1 election: Requested vote from peers 0748ca23dafe4fc88570234338ffdec1 (127.24.0.130:35331), 41ce5ec82b034993aaa0a6e2617407a7 (127.24.0.129:35121)
I20260812 01:21:38.991811 16381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f224233417d42b989c83b7df00df3f6" candidate_uuid: "0d48ccc189c44aa59f80d12affe2a109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41ce5ec82b034993aaa0a6e2617407a7"
I20260812 01:21:38.991880 16590 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f224233417d42b989c83b7df00df3f6" candidate_uuid: "0d48ccc189c44aa59f80d12affe2a109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0748ca23dafe4fc88570234338ffdec1"
I20260812 01:21:38.991896 16381 raft_consensus.cc:3037] T 4f224233417d42b989c83b7df00df3f6 P 41ce5ec82b034993aaa0a6e2617407a7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:21:38.991947 16590 raft_consensus.cc:3037] T 4f224233417d42b989c83b7df00df3f6 P 0748ca23dafe4fc88570234338ffdec1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:21:38.992760 16381 raft_consensus.cc:2454] T 4f224233417d42b989c83b7df00df3f6 P 41ce5ec82b034993aaa0a6e2617407a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d48ccc189c44aa59f80d12affe2a109 in term 1.
I20260812 01:21:38.992856 16590 raft_consensus.cc:2454] T 4f224233417d42b989c83b7df00df3f6 P 0748ca23dafe4fc88570234338ffdec1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d48ccc189c44aa59f80d12affe2a109 in term 1.
I20260812 01:21:38.992937 16694 leader_election.cc:302] T 4f224233417d42b989c83b7df00df3f6 P 0d48ccc189c44aa59f80d12affe2a109 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d48ccc189c44aa59f80d12affe2a109, 41ce5ec82b034993aaa0a6e2617407a7; no voters:
I20260812 01:21:38.993029 16896 raft_consensus.cc:2781] T 4f224233417d42b989c83b7df00df3f6 P 0d48ccc189c44aa59f80d12affe2a109 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:21:38.993194 16896 raft_consensus.cc:686] T 4f224233417d42b989c83b7df00df3f6 P 0d48ccc189c44aa59f80d12affe2a109 [term 1 LEADER]: Becoming Leader. State: Replica: 0d48ccc189c44aa59f80d12affe2a109, State: Running, Role: LEADER
I20260812 01:21:38.993297 16896 consensus_queue.cc:237] T 4f224233417d42b989c83b7df00df3f6 P 0d48ccc189c44aa59f80d12affe2a109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d48ccc189c44aa59f80d12affe2a109" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 43669 } } peers { permanent_uuid: "0748ca23dafe4fc88570234338ffdec1" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 35331 } } peers { permanent_uuid: "41ce5ec82b034993aaa0a6e2617407a7" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 35121 } }
I20260812 01:21:38.994113 16134 catalog_manager.cc:5495] T 4f224233417d42b989c83b7df00df3f6 P 0d48ccc189c44aa59f80d12affe2a109 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0d48ccc189c44aa59f80d12affe2a109 (127.24.0.131). New cstate: current_term: 1 leader_uuid: "0d48ccc189c44aa59f80d12affe2a109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d48ccc189c44aa59f80d12affe2a109" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 43669 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0748ca23dafe4fc88570234338ffdec1" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 35331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41ce5ec82b034993aaa0a6e2617407a7" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 35121 } health_report { overall_health: UNKNOWN } } }
I20260812 01:21:39.036123 16883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:39.033993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57688 (local address 127.24.0.131:43669)
0812 01:21:39.034062 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0812 01:21:39.034065 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:21:39.034076 (+ 11us) server_negotiation.cc:407] Connection header received
0812 01:21:39.034117 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:21:39.034117 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:21:39.034136 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:21:39.034195 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:21:39.034314 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:39.034774 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:39.035449 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:39.035625 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:39.035675 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:39.035796 (+ 121us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:21:39.035892 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:21:39.035953 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 01:21:39.035984 (+ 31us) server_negotiation.cc:299] Negotiation successful
0812 01:21:39.036000 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 01:21:39.037911 16590 raft_consensus.cc:1261] T 4f224233417d42b989c83b7df00df3f6 P 0748ca23dafe4fc88570234338ffdec1 [term 1 FOLLOWER]: Refusing update from remote peer 0d48ccc189c44aa59f80d12affe2a109: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:21:39.037959 16381 raft_consensus.cc:1261] T 4f224233417d42b989c83b7df00df3f6 P 41ce5ec82b034993aaa0a6e2617407a7 [term 1 FOLLOWER]: Refusing update from remote peer 0d48ccc189c44aa59f80d12affe2a109: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:21:39.038298 16900 consensus_queue.cc:1035] T 4f224233417d42b989c83b7df00df3f6 P 0d48ccc189c44aa59f80d12affe2a109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41ce5ec82b034993aaa0a6e2617407a7" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 35121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:21:39.038450 16896 consensus_queue.cc:1035] T 4f224233417d42b989c83b7df00df3f6 P 0d48ccc189c44aa59f80d12affe2a109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0748ca23dafe4fc88570234338ffdec1" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 35331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:21:39.040417 16905 mvcc.cc:205] Tried to move back new op lower bound from 7317494575256633344 to 7317494575077056512. Current Snapshot: MvccSnapshot[applied={T|T < 7317494575256633344}]
I20260812 01:21:39.042546 16137 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:54844:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:21:39.042657 16137 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:54844:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:21:39.043401 16137 catalog_manager.cc:5782] T 00000000000000000000000000000000 P eea6b06caffa4bfc8b5524bfacd0c7d3: Sending DeleteTablet for 3 replicas of tablet 4f224233417d42b989c83b7df00df3f6
I20260812 01:21:39.043733 16338 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f224233417d42b989c83b7df00df3f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:21:39 UTC) from {username='kudu', principal='kudu/127.24.0.190@KRBTEST.COM'} at 127.0.0.1:40176
I20260812 01:21:39.043764 16763 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f224233417d42b989c83b7df00df3f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:21:39 UTC) from {username='kudu', principal='kudu/127.24.0.190@KRBTEST.COM'} at 127.0.0.1:57676
I20260812 01:21:39.043762 16559 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f224233417d42b989c83b7df00df3f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:21:39 UTC) from {username='kudu', principal='kudu/127.24.0.190@KRBTEST.COM'} at 127.0.0.1:45576
I20260812 01:21:39.044078 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 16206
I20260812 01:21:39.044098 16922 tablet_replica.cc:330] T 4f224233417d42b989c83b7df00df3f6 P 0d48ccc189c44aa59f80d12affe2a109: stopping tablet replica
I20260812 01:21:39.044144 16919 tablet_replica.cc:330] T 4f224233417d42b989c83b7df00df3f6 P 41ce5ec82b034993aaa0a6e2617407a7: stopping tablet replica
I20260812 01:21:39.044189 16922 raft_consensus.cc:2229] T 4f224233417d42b989c83b7df00df3f6 P 0d48ccc189c44aa59f80d12affe2a109 [term 1 LEADER]: Raft consensus shutting down.
I20260812 01:21:39.044199 16919 raft_consensus.cc:2229] T 4f224233417d42b989c83b7df00df3f6 P 41ce5ec82b034993aaa0a6e2617407a7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 01:21:39.044245 16919 raft_consensus.cc:2258] T 4f224233417d42b989c83b7df00df3f6 P 41ce5ec82b034993aaa0a6e2617407a7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:21:39.044278 16922 raft_consensus.cc:2258] T 4f224233417d42b989c83b7df00df3f6 P 0d48ccc189c44aa59f80d12affe2a109 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:21:39.044414 16921 tablet_replica.cc:330] T 4f224233417d42b989c83b7df00df3f6 P 0748ca23dafe4fc88570234338ffdec1: stopping tablet replica
I20260812 01:21:39.044499 16921 raft_consensus.cc:2229] T 4f224233417d42b989c83b7df00df3f6 P 0748ca23dafe4fc88570234338ffdec1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 01:21:39.044590 16921 raft_consensus.cc:2258] T 4f224233417d42b989c83b7df00df3f6 P 0748ca23dafe4fc88570234338ffdec1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:21:39.064894 16921 ts_tablet_manager.cc:1905] T 4f224233417d42b989c83b7df00df3f6 P 0748ca23dafe4fc88570234338ffdec1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:21:39.067129 16921 ts_tablet_manager.cc:1918] T 4f224233417d42b989c83b7df00df3f6 P 0748ca23dafe4fc88570234338ffdec1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 01:21:39.067234 16921 log.cc:1196] T 4f224233417d42b989c83b7df00df3f6 P 0748ca23dafe4fc88570234338ffdec1: Deleting WAL directory at /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-1/wal/wals/4f224233417d42b989c83b7df00df3f6
I20260812 01:21:39.067529 16921 ts_tablet_manager.cc:1939] T 4f224233417d42b989c83b7df00df3f6 P 0748ca23dafe4fc88570234338ffdec1: Deleting consensus metadata
I20260812 01:21:39.068113 16112 catalog_manager.cc:4841] TS 0748ca23dafe4fc88570234338ffdec1 (127.24.0.130:35331): tablet 4f224233417d42b989c83b7df00df3f6 (table test-table [id=0cd6e8c79b8e424d9fd195df42ab565b]) successfully deleted
I20260812 01:21:39.068256 16922 ts_tablet_manager.cc:1905] T 4f224233417d42b989c83b7df00df3f6 P 0d48ccc189c44aa59f80d12affe2a109: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:21:39.070262 16922 ts_tablet_manager.cc:1918] T 4f224233417d42b989c83b7df00df3f6 P 0d48ccc189c44aa59f80d12affe2a109: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 01:21:39.070345 16922 log.cc:1196] T 4f224233417d42b989c83b7df00df3f6 P 0d48ccc189c44aa59f80d12affe2a109: Deleting WAL directory at /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-2/wal/wals/4f224233417d42b989c83b7df00df3f6
I20260812 01:21:39.070621 16922 ts_tablet_manager.cc:1939] T 4f224233417d42b989c83b7df00df3f6 P 0d48ccc189c44aa59f80d12affe2a109: Deleting consensus metadata
I20260812 01:21:39.071102 16112 catalog_manager.cc:4841] TS 0d48ccc189c44aa59f80d12affe2a109 (127.24.0.131:43669): tablet 4f224233417d42b989c83b7df00df3f6 (table test-table [id=0cd6e8c79b8e424d9fd195df42ab565b]) successfully deleted
W20260812 01:21:39.071494 16109 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.0.129:35121 (error 108)
I20260812 01:21:39.071981 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 16454
I20260812 01:21:39.072012 16884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:39.071866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.129:35121 (local address 127.0.0.1:40190)
0812 01:21:39.071920 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:39.071972 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.129:35121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260812 01:21:39.083042 16109 catalog_manager.cc:4584] TS 41ce5ec82b034993aaa0a6e2617407a7 (127.24.0.129:35121): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4f224233417d42b989c83b7df00df3f6: Network error: Client connection negotiation failed: client connection to 127.24.0.129:35121: connect: Connection refused (error 111)
I20260812 01:21:39.097865 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 16669
I20260812 01:21:39.109351 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 16081
I20260812 01:21:39.118345 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.190:38787
--webserver_interface=127.24.0.190
--webserver_port=45909
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab
--principal=kudu/127.24.0.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:33767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.0.190:38787
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:21:39.200865 16924 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:21:39.201102 16924 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:21:39.201140 16924 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:21:39.202265 16924 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:21:39.202337 16924 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:21:39.202363 16924 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:21:39.202384 16924 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:21:39.202412 16924 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:21:39.203866 16924 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:33767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.0.190:38787
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab
--principal=kudu/127.24.0.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.190:38787
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.0.190
--webserver_port=45909
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16924
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:21:39.204173 16924 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:21:39.204428 16924 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:21:39.209297 16938 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:21:39.209308 16933 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:21:39.209362 16932 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:21:39.209705 16924 server_base.cc:1034] running on GCE node
I20260812 01:21:39.209976 16924 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:21:39.210263 16924 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:21:39.211439 16924 hybrid_clock.cc:648] HybridClock initialized: now 1786497699211417 us; error 35 us; skew 500 ppm
Aug 12 01:21:39 dist-test-slave-t3q3 krb5kdc[15138](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497699, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:21:39.214071 16924 init.cc:377] Logged in from keytab as kudu/127.24.0.190@KRBTEST.COM (short username kudu)
I20260812 01:21:39.214504 16924 webserver.cc:460] Webserver started at http://127.24.0.190:45909/ using document root <none> and password file <none>
I20260812 01:21:39.214756 16924 fs_manager.cc:359] Metadata directory not provided
I20260812 01:21:39.214828 16924 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:21:39.216182 16924 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:21:39.216830 16951 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:21:39.217005 16924 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 01:21:39.217111 16924 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/master-0/wal
uuid: "eea6b06caffa4bfc8b5524bfacd0c7d3"
format_stamp: "Formatted at 2026-08-12 01:21:37 on dist-test-slave-t3q3"
server_key: "f819090b3b06392cede4af5a071fe241"
server_key_iv: "36a9b49b4a3a8d53137fed665012b789"
server_key_version: "encryptionkey@0"
I20260812 01:21:39.217449 16924 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:21:39.231333 16924 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:21:39.233213 16924 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:21:39.233371 16924 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:21:39.237459 16924 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.190:38787
I20260812 01:21:39.237530 17039 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.190:38787 every 8 connection(s)
I20260812 01:21:39.237897 16924 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
I20260812 01:21:39.239420 17040 sys_catalog.cc:263] Verifying existing consensus state
I20260812 01:21:39.240163 17040 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eea6b06caffa4bfc8b5524bfacd0c7d3: Bootstrap starting.
I20260812 01:21:39.244205 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 16924
I20260812 01:21:39.244472 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.0.129:35121
--local_ip_for_outbound_sockets=127.24.0.129
--tserver_master_addrs=127.24.0.190:38787
--webserver_port=33399
--webserver_interface=127.24.0.129
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.129.keytab
--principal=kudu/127.24.0.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:33767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 01:21:39.249822 17040 log.cc:824] T 00000000000000000000000000000000 P eea6b06caffa4bfc8b5524bfacd0c7d3: Log is configured to *not* fsync() on all Append() calls
I20260812 01:21:39.252199 17040 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eea6b06caffa4bfc8b5524bfacd0c7d3: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 01:21:39.252525 17040 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eea6b06caffa4bfc8b5524bfacd0c7d3: Bootstrap complete.
I20260812 01:21:39.254835 17040 raft_consensus.cc:348] T 00000000000000000000000000000000 P eea6b06caffa4bfc8b5524bfacd0c7d3 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eea6b06caffa4bfc8b5524bfacd0c7d3" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 38787 } }
I20260812 01:21:39.255054 17040 raft_consensus.cc:729] T 00000000000000000000000000000000 P eea6b06caffa4bfc8b5524bfacd0c7d3 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: eea6b06caffa4bfc8b5524bfacd0c7d3, State: Initialized, Role: FOLLOWER
I20260812 01:21:39.255219 17040 consensus_queue.cc:260] T 00000000000000000000000000000000 P eea6b06caffa4bfc8b5524bfacd0c7d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eea6b06caffa4bfc8b5524bfacd0c7d3" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 38787 } }
I20260812 01:21:39.255291 17040 raft_consensus.cc:388] T 00000000000000000000000000000000 P eea6b06caffa4bfc8b5524bfacd0c7d3 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:21:39.255353 17040 raft_consensus.cc:482] T 00000000000000000000000000000000 P eea6b06caffa4bfc8b5524bfacd0c7d3 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:21:39.255400 17040 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eea6b06caffa4bfc8b5524bfacd0c7d3 [term 2 FOLLOWER]: Advancing to term 3
I20260812 01:21:39.256479 17040 raft_consensus.cc:504] T 00000000000000000000000000000000 P eea6b06caffa4bfc8b5524bfacd0c7d3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eea6b06caffa4bfc8b5524bfacd0c7d3" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 38787 } }
I20260812 01:21:39.256592 17040 leader_election.cc:302] T 00000000000000000000000000000000 P eea6b06caffa4bfc8b5524bfacd0c7d3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eea6b06caffa4bfc8b5524bfacd0c7d3; no voters:
I20260812 01:21:39.256809 17040 leader_election.cc:288] T 00000000000000000000000000000000 P eea6b06caffa4bfc8b5524bfacd0c7d3 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260812 01:21:39.256843 17050 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eea6b06caffa4bfc8b5524bfacd0c7d3 [term 3 FOLLOWER]: Leader election won for term 3
I20260812 01:21:39.256959 17050 raft_consensus.cc:686] T 00000000000000000000000000000000 P eea6b06caffa4bfc8b5524bfacd0c7d3 [term 3 LEADER]: Becoming Leader. State: Replica: eea6b06caffa4bfc8b5524bfacd0c7d3, State: Running, Role: LEADER
I20260812 01:21:39.257098 17040 sys_catalog.cc:564] T 00000000000000000000000000000000 P eea6b06caffa4bfc8b5524bfacd0c7d3 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:21:39.257129 17050 consensus_queue.cc:237] T 00000000000000000000000000000000 P eea6b06caffa4bfc8b5524bfacd0c7d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eea6b06caffa4bfc8b5524bfacd0c7d3" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 38787 } }
I20260812 01:21:39.257592 17052 sys_catalog.cc:455] T 00000000000000000000000000000000 P eea6b06caffa4bfc8b5524bfacd0c7d3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "eea6b06caffa4bfc8b5524bfacd0c7d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eea6b06caffa4bfc8b5524bfacd0c7d3" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 38787 } } }
I20260812 01:21:39.257606 17053 sys_catalog.cc:455] T 00000000000000000000000000000000 P eea6b06caffa4bfc8b5524bfacd0c7d3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eea6b06caffa4bfc8b5524bfacd0c7d3. Latest consensus state: current_term: 3 leader_uuid: "eea6b06caffa4bfc8b5524bfacd0c7d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eea6b06caffa4bfc8b5524bfacd0c7d3" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 38787 } } }
I20260812 01:21:39.257732 17053 sys_catalog.cc:458] T 00000000000000000000000000000000 P eea6b06caffa4bfc8b5524bfacd0c7d3 [sys.catalog]: This master's current role is: LEADER
I20260812 01:21:39.257688 17052 sys_catalog.cc:458] T 00000000000000000000000000000000 P eea6b06caffa4bfc8b5524bfacd0c7d3 [sys.catalog]: This master's current role is: LEADER
I20260812 01:21:39.258252 17064 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:21:39.259262 17064 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:21:39.259465 17064 catalog_manager.cc:1259] Loaded cluster ID: 5434671b92c84d6294600128656e9035
I20260812 01:21:39.259575 17064 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260812 01:21:39.260089 17064 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f41610859ad google::LogMessage::Fail() at ??:0
@ 0x7f416108888a google::LogMessage::SendToLog() at ??:0
@ 0x7f41610854de google::LogMessage::Flush() at ??:0
@ 0x7f4161086a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f4168fa60f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f4168faf8b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f4162f530d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f4162f48954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f4163dac6db start_thread at ??:0
@ 0x7f415fb8361f clone at ??:0
W20260812 01:21:39.344405 17046 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:21:39.344607 17046 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:21:39.344630 17046 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:21:39.345772 17046 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:21:39.345813 17046 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:21:39.345839 17046 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.0.129
I20260812 01:21:39.347293 17046 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:33767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.129.keytab
--principal=kudu/127.24.0.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.0.129:35121
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.0.129
--webserver_port=33399
--tserver_master_addrs=127.24.0.190:38787
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17046
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.0.129
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:21:39.347543 17046 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:21:39.347790 17046 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:21:39.353045 17120 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:21:39.353034 17119 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:21:39.353036 17128 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:21:39.353287 17046 server_base.cc:1034] running on GCE node
I20260812 01:21:39.353479 17046 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:21:39.353677 17046 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:21:39.354842 17046 hybrid_clock.cc:648] HybridClock initialized: now 1786497699354827 us; error 30 us; skew 500 ppm
Aug 12 01:21:39 dist-test-slave-t3q3 krb5kdc[15138](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497699, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:21:39.357609 17046 init.cc:377] Logged in from keytab as kudu/127.24.0.129@KRBTEST.COM (short username kudu)
I20260812 01:21:39.358098 17046 webserver.cc:460] Webserver started at http://127.24.0.129:33399/ using document root <none> and password file <none>
I20260812 01:21:39.358352 17046 fs_manager.cc:359] Metadata directory not provided
I20260812 01:21:39.358399 17046 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:21:39.359723 17046 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:21:39.360483 17143 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:21:39.360636 17046 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 01:21:39.360743 17046 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-0/wal
uuid: "41ce5ec82b034993aaa0a6e2617407a7"
format_stamp: "Formatted at 2026-08-12 01:21:37 on dist-test-slave-t3q3"
server_key: "2703cfff634566d3398a91cf0eb6bd6c"
server_key_iv: "f00c7bc63e81c634d05b20bbe7f55b33"
server_key_version: "encryptionkey@0"
I20260812 01:21:39.361091 17046 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:21:39.374152 17046 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:21:39.376557 17046 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:21:39.376729 17046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:21:39.376955 17046 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:21:39.377453 17155 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 01:21:39.379395 17046 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 01:21:39.379455 17046 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260812 01:21:39.379499 17046 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 01:21:39.380210 17046 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 01:21:39.380263 17046 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.001s sys 0.000s
I20260812 01:21:39.380364 17155 tablet_bootstrap.cc:492] T 4f224233417d42b989c83b7df00df3f6 P 41ce5ec82b034993aaa0a6e2617407a7: Bootstrap starting.
I20260812 01:21:39.387921 17046 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.129:35121
I20260812 01:21:39.387993 17351 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.129:35121 every 8 connection(s)
I20260812 01:21:39.388423 17046 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-0/data/info.pb
I20260812 01:21:39.392148 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 17046
I20260812 01:21:39.392455 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.0.130:35331
--local_ip_for_outbound_sockets=127.24.0.130
--tserver_master_addrs=127.24.0.190:38787
--webserver_port=40319
--webserver_interface=127.24.0.130
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.130.keytab
--principal=kudu/127.24.0.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:33767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 01:21:39.392920 17155 log.cc:824] T 4f224233417d42b989c83b7df00df3f6 P 41ce5ec82b034993aaa0a6e2617407a7: Log is configured to *not* fsync() on all Append() calls
I20260812 01:21:39.396188 17155 tablet_bootstrap.cc:492] T 4f224233417d42b989c83b7df00df3f6 P 41ce5ec82b034993aaa0a6e2617407a7: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 01:21:39.396615 17155 tablet_bootstrap.cc:492] T 4f224233417d42b989c83b7df00df3f6 P 41ce5ec82b034993aaa0a6e2617407a7: Bootstrap complete.
I20260812 01:21:39.397935 17155 ts_tablet_manager.cc:1397] T 4f224233417d42b989c83b7df00df3f6 P 41ce5ec82b034993aaa0a6e2617407a7: Time spent bootstrapping tablet: real 0.018s user 0.002s sys 0.013s
I20260812 01:21:39.399600 17155 raft_consensus.cc:348] T 4f224233417d42b989c83b7df00df3f6 P 41ce5ec82b034993aaa0a6e2617407a7 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d48ccc189c44aa59f80d12affe2a109" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 43669 } } peers { permanent_uuid: "0748ca23dafe4fc88570234338ffdec1" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 35331 } } peers { permanent_uuid: "41ce5ec82b034993aaa0a6e2617407a7" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 35121 } }
I20260812 01:21:39.400009 17155 raft_consensus.cc:729] T 4f224233417d42b989c83b7df00df3f6 P 41ce5ec82b034993aaa0a6e2617407a7 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41ce5ec82b034993aaa0a6e2617407a7, State: Initialized, Role: FOLLOWER
I20260812 01:21:39.400269 17155 consensus_queue.cc:260] T 4f224233417d42b989c83b7df00df3f6 P 41ce5ec82b034993aaa0a6e2617407a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d48ccc189c44aa59f80d12affe2a109" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 43669 } } peers { permanent_uuid: "0748ca23dafe4fc88570234338ffdec1" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 35331 } } peers { permanent_uuid: "41ce5ec82b034993aaa0a6e2617407a7" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 35121 } }
I20260812 01:21:39.400625 17155 ts_tablet_manager.cc:1428] T 4f224233417d42b989c83b7df00df3f6 P 41ce5ec82b034993aaa0a6e2617407a7: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
W20260812 01:21:39.478956 17356 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:21:39.479172 17356 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:21:39.479210 17356 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:21:39.480417 17356 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:21:39.480475 17356 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:21:39.480521 17356 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.0.130
I20260812 01:21:39.481945 17356 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:33767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.130.keytab
--principal=kudu/127.24.0.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.0.130:35331
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.0.130
--webserver_port=40319
--tserver_master_addrs=127.24.0.190:38787
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17356
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.0.130
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:21:39.482203 17356 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:21:39.482439 17356 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:21:39.487546 17375 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:21:39.487557 17367 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:21:39.487747 17356 server_base.cc:1034] running on GCE node
W20260812 01:21:39.487557 17368 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:21:39.488117 17356 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:21:39.488425 17356 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:21:39.489576 17356 hybrid_clock.cc:648] HybridClock initialized: now 1786497699489558 us; error 34 us; skew 500 ppm
Aug 12 01:21:39 dist-test-slave-t3q3 krb5kdc[15138](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497699, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:21:39.492288 17356 init.cc:377] Logged in from keytab as kudu/127.24.0.130@KRBTEST.COM (short username kudu)
I20260812 01:21:39.492808 17356 webserver.cc:460] Webserver started at http://127.24.0.130:40319/ using document root <none> and password file <none>
I20260812 01:21:39.493062 17356 fs_manager.cc:359] Metadata directory not provided
I20260812 01:21:39.493134 17356 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:21:39.494510 17356 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:21:39.495154 17389 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:21:39.495318 17356 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:21:39.495405 17356 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-1/wal
uuid: "0748ca23dafe4fc88570234338ffdec1"
format_stamp: "Formatted at 2026-08-12 01:21:37 on dist-test-slave-t3q3"
server_key: "5f044f54d62149cb82904befdb3ad5cb"
server_key_iv: "5fd9fa8d22942fcd892eee45cda85e51"
server_key_version: "encryptionkey@0"
I20260812 01:21:39.495720 17356 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
W20260812 01:21:39.509438 17354 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:21:39.389056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:38787 (local address 127.24.0.129:37059)
0812 01:21:39.389241 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:39.389284 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 01:21:39.389738 (+ 454us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:21:39.509239 (+119501us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:38787: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":110,"threads_started":1}
W20260812 01:21:39.509681 17352 heartbeater.cc:644] Failed to heartbeat to 127.24.0.190:38787 (0 consecutive failures): Network error: Failed to ping master at 127.24.0.190:38787: Client connection negotiation failed: client connection to 127.24.0.190:38787: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 01:21:39.510040 17354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:39.509929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:38787 (local address 127.24.0.129:38857)
0812 01:21:39.509982 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:39.510022 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260812 01:21:39.510370 17354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:39.510275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:38787 (local address 127.24.0.129:36769)
0812 01:21:39.510322 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:39.510353 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260812 01:21:39.510480 17352 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 01:21:39.510671 17356 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:21:39.512660 17356 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:21:39.512831 17356 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:21:39.513077 17356 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:21:39.513396 17356 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:21:39.513449 17356 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:21:39.513491 17356 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:21:39.513520 17356 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:21:39.519726 17356 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.130:35331
I20260812 01:21:39.519783 17582 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.130:35331 every 8 connection(s)
I20260812 01:21:39.520169 17356 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-1/data/info.pb
I20260812 01:21:39.521374 17588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:39.520836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:38787 (local address 127.24.0.130:44683)
0812 01:21:39.521068 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:39.521147 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":89,"threads_started":1}
W20260812 01:21:39.521837 17583 heartbeater.cc:644] Failed to heartbeat to 127.24.0.190:38787 (0 consecutive failures): Network error: Failed to ping master at 127.24.0.190:38787: Client connection negotiation failed: client connection to 127.24.0.190:38787: connect: Connection refused (error 111)
I20260812 01:21:39.522141 17588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:39.522047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:38787 (local address 127.24.0.130:51307)
0812 01:21:39.522094 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:39.522123 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260812 01:21:39.522418 17588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:39.522368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:38787 (local address 127.24.0.130:36397)
0812 01:21:39.522388 (+ 20us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:39.522406 (+ 18us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":11}
W20260812 01:21:39.522547 17583 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 01:21:39.528929 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 17356
I20260812 01:21:39.529191 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.0.131:43669
--local_ip_for_outbound_sockets=127.24.0.131
--tserver_master_addrs=127.24.0.190:38787
--webserver_port=42015
--webserver_interface=127.24.0.131
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.131.keytab
--principal=kudu/127.24.0.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:33767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:21:39.612239 17590 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:21:39.612424 17590 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:21:39.612447 17590 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:21:39.613595 17590 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:21:39.613638 17590 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:21:39.613664 17590 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.0.131
I20260812 01:21:39.615123 17590 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:33767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.131.keytab
--principal=kudu/127.24.0.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.0.131:43669
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.0.131
--webserver_port=42015
--tserver_master_addrs=127.24.0.190:38787
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17590
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.0.131
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:21:39.615366 17590 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:21:39.615607 17590 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:21:39.620482 17598 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:21:39.620489 17601 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:21:39.620489 17599 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:21:39.620853 17590 server_base.cc:1034] running on GCE node
I20260812 01:21:39.621021 17590 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:21:39.621210 17590 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:21:39.622385 17590 hybrid_clock.cc:648] HybridClock initialized: now 1786497699622369 us; error 29 us; skew 500 ppm
Aug 12 01:21:39 dist-test-slave-t3q3 krb5kdc[15138](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497699, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:21:39.624869 17590 init.cc:377] Logged in from keytab as kudu/127.24.0.131@KRBTEST.COM (short username kudu)
I20260812 01:21:39.625331 17590 webserver.cc:460] Webserver started at http://127.24.0.131:42015/ using document root <none> and password file <none>
I20260812 01:21:39.625573 17590 fs_manager.cc:359] Metadata directory not provided
I20260812 01:21:39.625653 17590 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:21:39.626940 17590 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 01:21:39.627544 17614 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:21:39.627774 17590 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 01:21:39.627871 17590 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-2/wal
uuid: "0d48ccc189c44aa59f80d12affe2a109"
format_stamp: "Formatted at 2026-08-12 01:21:37 on dist-test-slave-t3q3"
server_key: "4e8e79c992c5758da2ada0484c88e2bb"
server_key_iv: "b1bd463be88d537e77a43a1b4ef41879"
server_key_version: "encryptionkey@0"
I20260812 01:21:39.628271 17590 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:21:39.641978 17590 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:21:39.643731 17590 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:21:39.643884 17590 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:21:39.644170 17590 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:21:39.644488 17590 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:21:39.644543 17590 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:21:39.644575 17590 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:21:39.644603 17590 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:21:39.650418 17590 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.131:43669
I20260812 01:21:39.650497 17795 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.131:43669 every 8 connection(s)
I20260812 01:21:39.650846 17590 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786497534195564-24578-0/minicluster-data/ts-2/data/info.pb
I20260812 01:21:39.651911 17799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:39.651537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:38787 (local address 127.24.0.131:41739)
0812 01:21:39.651697 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:39.651775 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":77,"threads_started":1}
W20260812 01:21:39.652341 17796 heartbeater.cc:644] Failed to heartbeat to 127.24.0.190:38787 (0 consecutive failures): Network error: Failed to ping master at 127.24.0.190:38787: Client connection negotiation failed: client connection to 127.24.0.190:38787: connect: Connection refused (error 111)
I20260812 01:21:39.652607 17799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:39.652517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:38787 (local address 127.24.0.131:46157)
0812 01:21:39.652563 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:39.652590 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260812 01:21:39.652877 17799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:39.652795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:38787 (local address 127.24.0.131:47997)
0812 01:21:39.652836 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:39.652864 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260812 01:21:39.653000 17796 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 01:21:39.654582 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 17590
I20260812 01:21:40.511088 17803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:40.510850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:38787 (local address 127.24.0.129:37149)
0812 01:21:40.510984 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:40.511023 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":67,"threads_started":1}
I20260812 01:21:40.523070 17804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:40.522871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:38787 (local address 127.24.0.130:48799)
0812 01:21:40.522982 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:40.523019 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":59,"threads_started":1}
I20260812 01:21:40.653590 17805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:40.653359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:38787 (local address 127.24.0.131:49175)
0812 01:21:40.653500 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:40.653542 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":63,"threads_started":1}
I20260812 01:21:40.833405 17806 raft_consensus.cc:482] T 4f224233417d42b989c83b7df00df3f6 P 41ce5ec82b034993aaa0a6e2617407a7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 01:21:40.833568 17806 raft_consensus.cc:504] T 4f224233417d42b989c83b7df00df3f6 P 41ce5ec82b034993aaa0a6e2617407a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d48ccc189c44aa59f80d12affe2a109" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 43669 } } peers { permanent_uuid: "0748ca23dafe4fc88570234338ffdec1" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 35331 } } peers { permanent_uuid: "41ce5ec82b034993aaa0a6e2617407a7" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 35121 } }
I20260812 01:21:40.833920 17806 leader_election.cc:288] T 4f224233417d42b989c83b7df00df3f6 P 41ce5ec82b034993aaa0a6e2617407a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0d48ccc189c44aa59f80d12affe2a109 (127.24.0.131:43669), 0748ca23dafe4fc88570234338ffdec1 (127.24.0.130:35331)
Aug 12 01:21:40 dist-test-slave-t3q3 krb5kdc[15138](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497699, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.129@KRBTEST.COM for kudu/127.24.0.131@KRBTEST.COM
Aug 12 01:21:40 dist-test-slave-t3q3 krb5kdc[15138](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497699, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.129@KRBTEST.COM for kudu/127.24.0.130@KRBTEST.COM
I20260812 01:21:40.845973 17807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:40.834294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.129:52961 (local address 127.24.0.131:43669)
0812 01:21:40.834522 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0812 01:21:40.834532 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 01:21:40.834596 (+ 64us) server_negotiation.cc:407] Connection header received
0812 01:21:40.835228 (+ 632us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:21:40.835232 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:21:40.835293 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:21:40.835455 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:21:40.836469 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:40.837868 (+ 1399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:40.839401 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:40.840053 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:40.840196 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:40.842497 (+ 2301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:21:40.842501 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:21:40.842502 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:21:40.842513 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:21:40.844323 (+ 1810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:21:40.844738 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:21:40.844742 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:21:40.844742 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:21:40.844834 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:21:40.845093 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:21:40.845094 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:21:40.845094 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:21:40.845332 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:21:40.845437 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0812 01:21:40.845733 (+ 296us) server_negotiation.cc:299] Negotiation successful
0812 01:21:40.845854 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":61,"threads_started":1}
I20260812 01:21:40.846014 17803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:40.834180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.131:43669 (local address 127.24.0.129:52961)
0812 01:21:40.834261 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:40.834279 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 01:21:40.834358 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:21:40.835524 (+ 1166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:21:40.835527 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:21:40.836038 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:21:40.836336 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:21:40.836343 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:21:40.837986 (+ 1643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:21:40.837986 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:21:40.839289 (+ 1303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:21:40.839292 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:21:40.840166 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:21:40.840167 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:21:40.840265 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:40.840842 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:21:40.840853 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:21:40.842373 (+ 1520us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:21:40.844409 (+ 2036us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:21:40.844415 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:21:40.844418 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:21:40.844652 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:21:40.844915 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:21:40.844915 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:21:40.844916 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:21:40.845027 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:21:40.845418 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:21:40.845425 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:21:40.845561 (+ 136us) client_negotiation.cc:769] Sending connection context
0812 01:21:40.845676 (+ 115us) client_negotiation.cc:240] Negotiation successful
0812 01:21:40.845846 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55,"mutex_wait_us":82}
I20260812 01:21:40.846120 17809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:40.834481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.129:39315 (local address 127.24.0.130:35331)
0812 01:21:40.834649 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0812 01:21:40.834657 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 01:21:40.834690 (+ 33us) server_negotiation.cc:407] Connection header received
0812 01:21:40.835228 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:21:40.835232 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:21:40.835293 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:21:40.835455 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:21:40.836470 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:40.837868 (+ 1398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:40.839401 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:40.840054 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:40.840197 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:40.842842 (+ 2645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:21:40.842845 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:21:40.842846 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:21:40.842878 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:21:40.844683 (+ 1805us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:21:40.845062 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:21:40.845065 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:21:40.845066 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:21:40.845156 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:21:40.845376 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:21:40.845377 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:21:40.845377 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:21:40.845599 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:21:40.845686 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 01:21:40.845881 (+ 195us) server_negotiation.cc:299] Negotiation successful
0812 01:21:40.846013 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":59,"threads_started":1}
I20260812 01:21:40.846642 17808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:40.834346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.130:35331 (local address 127.24.0.129:39315)
0812 01:21:40.834452 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:40.834464 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 01:21:40.834513 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:21:40.835524 (+ 1011us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:21:40.835527 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:21:40.836038 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:21:40.836335 (+ 297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:21:40.836343 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:21:40.837983 (+ 1640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:21:40.837984 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:21:40.839289 (+ 1305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:21:40.839292 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:21:40.840318 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:21:40.840319 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:21:40.840369 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:40.840989 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:21:40.840993 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:21:40.842728 (+ 1735us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:21:40.844802 (+ 2074us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:21:40.844803 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:21:40.844804 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:21:40.844978 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:21:40.845240 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:21:40.845241 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:21:40.845241 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:21:40.845306 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:21:40.845682 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:21:40.845683 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:21:40.845729 (+ 46us) client_negotiation.cc:769] Sending connection context
0812 01:21:40.845817 (+ 88us) client_negotiation.cc:240] Negotiation successful
0812 01:21:40.845920 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93,"mutex_wait_us":51,"thread_start_us":59,"threads_started":1}
W20260812 01:21:40.847101 17148 leader_election.cc:341] T 4f224233417d42b989c83b7df00df3f6 P 41ce5ec82b034993aaa0a6e2617407a7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0d48ccc189c44aa59f80d12affe2a109 (127.24.0.131:43669): Not found: Tablet not found: 4f224233417d42b989c83b7df00df3f6
W20260812 01:21:40.847330 17148 leader_election.cc:341] T 4f224233417d42b989c83b7df00df3f6 P 41ce5ec82b034993aaa0a6e2617407a7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0748ca23dafe4fc88570234338ffdec1 (127.24.0.130:35331): Not found: Tablet not found: 4f224233417d42b989c83b7df00df3f6
I20260812 01:21:40.847381 17148 leader_election.cc:302] T 4f224233417d42b989c83b7df00df3f6 P 41ce5ec82b034993aaa0a6e2617407a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41ce5ec82b034993aaa0a6e2617407a7; no voters: 0748ca23dafe4fc88570234338ffdec1, 0d48ccc189c44aa59f80d12affe2a109
I20260812 01:21:40.847496 17806 raft_consensus.cc:2726] T 4f224233417d42b989c83b7df00df3f6 P 41ce5ec82b034993aaa0a6e2617407a7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 01:21:41.512008 17812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:41.511751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:38787 (local address 127.24.0.129:33209)
0812 01:21:41.511908 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:41.511959 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":72,"threads_started":1}
I20260812 01:21:41.523789 17813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:41.523585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:38787 (local address 127.24.0.130:56707)
0812 01:21:41.523712 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:41.523755 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":63,"threads_started":1}
I20260812 01:21:41.654232 17815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:41.654013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:38787 (local address 127.24.0.131:34989)
0812 01:21:41.654147 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:41.654188 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":63,"threads_started":1}
I20260812 01:21:42.512794 17817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:42.512530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:38787 (local address 127.24.0.129:49421)
0812 01:21:42.512708 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:42.512745 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":89,"threads_started":1}
I20260812 01:21:42.524478 17818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:42.524293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:38787 (local address 127.24.0.130:47465)
0812 01:21:42.524409 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:42.524441 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":59,"threads_started":1}
I20260812 01:21:42.654929 17819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:42.654714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:38787 (local address 127.24.0.131:55481)
0812 01:21:42.654849 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:42.654887 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":67,"threads_started":1}
I20260812 01:21:42.660583 17822 raft_consensus.cc:482] T 4f224233417d42b989c83b7df00df3f6 P 41ce5ec82b034993aaa0a6e2617407a7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 01:21:42.660692 17822 raft_consensus.cc:504] T 4f224233417d42b989c83b7df00df3f6 P 41ce5ec82b034993aaa0a6e2617407a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d48ccc189c44aa59f80d12affe2a109" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 43669 } } peers { permanent_uuid: "0748ca23dafe4fc88570234338ffdec1" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 35331 } } peers { permanent_uuid: "41ce5ec82b034993aaa0a6e2617407a7" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 35121 } }
I20260812 01:21:42.660874 17822 leader_election.cc:288] T 4f224233417d42b989c83b7df00df3f6 P 41ce5ec82b034993aaa0a6e2617407a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0d48ccc189c44aa59f80d12affe2a109 (127.24.0.131:43669), 0748ca23dafe4fc88570234338ffdec1 (127.24.0.130:35331)
W20260812 01:21:42.661293 17148 leader_election.cc:341] T 4f224233417d42b989c83b7df00df3f6 P 41ce5ec82b034993aaa0a6e2617407a7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0d48ccc189c44aa59f80d12affe2a109 (127.24.0.131:43669): Not found: Tablet not found: 4f224233417d42b989c83b7df00df3f6
W20260812 01:21:42.661391 17148 leader_election.cc:341] T 4f224233417d42b989c83b7df00df3f6 P 41ce5ec82b034993aaa0a6e2617407a7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0748ca23dafe4fc88570234338ffdec1 (127.24.0.130:35331): Not found: Tablet not found: 4f224233417d42b989c83b7df00df3f6
I20260812 01:21:42.661427 17148 leader_election.cc:302] T 4f224233417d42b989c83b7df00df3f6 P 41ce5ec82b034993aaa0a6e2617407a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41ce5ec82b034993aaa0a6e2617407a7; no voters: 0748ca23dafe4fc88570234338ffdec1, 0d48ccc189c44aa59f80d12affe2a109
I20260812 01:21:42.661526 17822 raft_consensus.cc:2726] T 4f224233417d42b989c83b7df00df3f6 P 41ce5ec82b034993aaa0a6e2617407a7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 01:21:43.513481 17825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:43.513264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:38787 (local address 127.24.0.129:36587)
0812 01:21:43.513398 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:43.513434 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":64,"threads_started":1}
I20260812 01:21:43.525097 17826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:43.524898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:38787 (local address 127.24.0.130:52471)
0812 01:21:43.525033 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:43.525061 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":88,"threads_started":1}
I20260812 01:21:43.655526 17830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:43.655330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:38787 (local address 127.24.0.131:46261)
0812 01:21:43.655454 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:43.655489 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":63,"threads_started":1}
I20260812 01:21:44.223747 17831 raft_consensus.cc:482] T 4f224233417d42b989c83b7df00df3f6 P 41ce5ec82b034993aaa0a6e2617407a7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 01:21:44.223837 17831 raft_consensus.cc:504] T 4f224233417d42b989c83b7df00df3f6 P 41ce5ec82b034993aaa0a6e2617407a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d48ccc189c44aa59f80d12affe2a109" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 43669 } } peers { permanent_uuid: "0748ca23dafe4fc88570234338ffdec1" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 35331 } } peers { permanent_uuid: "41ce5ec82b034993aaa0a6e2617407a7" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 35121 } }
I20260812 01:21:44.224040 17831 leader_election.cc:288] T 4f224233417d42b989c83b7df00df3f6 P 41ce5ec82b034993aaa0a6e2617407a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0d48ccc189c44aa59f80d12affe2a109 (127.24.0.131:43669), 0748ca23dafe4fc88570234338ffdec1 (127.24.0.130:35331)
W20260812 01:21:44.224447 17148 leader_election.cc:341] T 4f224233417d42b989c83b7df00df3f6 P 41ce5ec82b034993aaa0a6e2617407a7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0d48ccc189c44aa59f80d12affe2a109 (127.24.0.131:43669): Not found: Tablet not found: 4f224233417d42b989c83b7df00df3f6
W20260812 01:21:44.224532 17148 leader_election.cc:341] T 4f224233417d42b989c83b7df00df3f6 P 41ce5ec82b034993aaa0a6e2617407a7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0748ca23dafe4fc88570234338ffdec1 (127.24.0.130:35331): Not found: Tablet not found: 4f224233417d42b989c83b7df00df3f6
I20260812 01:21:44.224550 17148 leader_election.cc:302] T 4f224233417d42b989c83b7df00df3f6 P 41ce5ec82b034993aaa0a6e2617407a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41ce5ec82b034993aaa0a6e2617407a7; no voters: 0748ca23dafe4fc88570234338ffdec1, 0d48ccc189c44aa59f80d12affe2a109
I20260812 01:21:44.224627 17831 raft_consensus.cc:2726] T 4f224233417d42b989c83b7df00df3f6 P 41ce5ec82b034993aaa0a6e2617407a7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 01:21:44.514256 17833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:44.514016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:38787 (local address 127.24.0.129:44991)
0812 01:21:44.514172 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:44.514209 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":86,"threads_started":1}
I20260812 01:21:44.525727 17835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:44.525519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:38787 (local address 127.24.0.130:32925)
0812 01:21:44.525639 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:44.525675 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":63,"threads_started":1}
I20260812 01:21:44.656221 17836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:44.655985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:38787 (local address 127.24.0.131:51777)
0812 01:21:44.656133 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:44.656180 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":64,"threads_started":1}
I20260812 01:21:45.514951 17837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:45.514731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:38787 (local address 127.24.0.129:40151)
0812 01:21:45.514868 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:45.514905 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":66,"threads_started":1}
I20260812 01:21:45.526398 17838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:45.526168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:38787 (local address 127.24.0.130:40625)
0812 01:21:45.526332 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:45.526363 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":101,"threads_started":1}
I20260812 01:21:45.656924 17839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:45.656708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:38787 (local address 127.24.0.131:34593)
0812 01:21:45.656844 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:45.656880 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":76,"threads_started":1}
I20260812 01:21:46.515601 17840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:46.515391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:38787 (local address 127.24.0.129:48695)
0812 01:21:46.515516 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:46.515555 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":64,"threads_started":1}
I20260812 01:21:46.527240 17841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:46.526965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:38787 (local address 127.24.0.130:42083)
0812 01:21:46.527128 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:46.527159 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":110,"threads_started":1}
I20260812 01:21:46.657617 17843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:46.657417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:38787 (local address 127.24.0.131:51469)
0812 01:21:46.657542 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:46.657576 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":65,"threads_started":1}
I20260812 01:21:47.516320 17846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:47.516037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:38787 (local address 127.24.0.129:50425)
0812 01:21:47.516238 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:47.516273 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":119,"threads_started":1}
I20260812 01:21:47.527915 17847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:47.527702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:38787 (local address 127.24.0.130:38079)
0812 01:21:47.527841 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:47.527877 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":93,"threads_started":1}
I20260812 01:21:47.658387 17849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:47.658157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:38787 (local address 127.24.0.131:47139)
0812 01:21:47.658300 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:47.658341 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":67,"threads_started":1}
I20260812 01:21:48.048262 17850 raft_consensus.cc:482] T 4f224233417d42b989c83b7df00df3f6 P 41ce5ec82b034993aaa0a6e2617407a7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 01:21:48.048393 17850 raft_consensus.cc:504] T 4f224233417d42b989c83b7df00df3f6 P 41ce5ec82b034993aaa0a6e2617407a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d48ccc189c44aa59f80d12affe2a109" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 43669 } } peers { permanent_uuid: "0748ca23dafe4fc88570234338ffdec1" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 35331 } } peers { permanent_uuid: "41ce5ec82b034993aaa0a6e2617407a7" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 35121 } }
I20260812 01:21:48.048547 17850 leader_election.cc:288] T 4f224233417d42b989c83b7df00df3f6 P 41ce5ec82b034993aaa0a6e2617407a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0d48ccc189c44aa59f80d12affe2a109 (127.24.0.131:43669), 0748ca23dafe4fc88570234338ffdec1 (127.24.0.130:35331)
W20260812 01:21:48.048918 17148 leader_election.cc:341] T 4f224233417d42b989c83b7df00df3f6 P 41ce5ec82b034993aaa0a6e2617407a7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0748ca23dafe4fc88570234338ffdec1 (127.24.0.130:35331): Not found: Tablet not found: 4f224233417d42b989c83b7df00df3f6
W20260812 01:21:48.049011 17148 leader_election.cc:341] T 4f224233417d42b989c83b7df00df3f6 P 41ce5ec82b034993aaa0a6e2617407a7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0d48ccc189c44aa59f80d12affe2a109 (127.24.0.131:43669): Not found: Tablet not found: 4f224233417d42b989c83b7df00df3f6
I20260812 01:21:48.049045 17148 leader_election.cc:302] T 4f224233417d42b989c83b7df00df3f6 P 41ce5ec82b034993aaa0a6e2617407a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41ce5ec82b034993aaa0a6e2617407a7; no voters: 0748ca23dafe4fc88570234338ffdec1, 0d48ccc189c44aa59f80d12affe2a109
I20260812 01:21:48.049170 17850 raft_consensus.cc:2726] T 4f224233417d42b989c83b7df00df3f6 P 41ce5ec82b034993aaa0a6e2617407a7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 01:21:48.517495 17851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:48.516834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:38787 (local address 127.24.0.129:46599)
0812 01:21:48.517407 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:48.517446 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":545,"thread_start_us":514,"threads_started":1}
I20260812 01:21:48.528666 17853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:48.528451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:38787 (local address 127.24.0.130:41599)
0812 01:21:48.528578 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:48.528621 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":62,"threads_started":1}
I20260812 01:21:48.658981 17855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:48.658792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:38787 (local address 127.24.0.131:38665)
0812 01:21:48.658911 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:48.658942 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":63,"threads_started":1}
I20260812 01:21:49.518200 17856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:49.517949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:38787 (local address 127.24.0.129:57109)
0812 01:21:49.518101 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:49.518154 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":87,"threads_started":1}
I20260812 01:21:49.529279 17858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:49.529094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:38787 (local address 127.24.0.130:49557)
0812 01:21:49.529211 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:49.529241 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":63,"threads_started":1}
I20260812 01:21:49.659644 17862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:49.659410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:38787 (local address 127.24.0.131:43037)
0812 01:21:49.659559 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:49.659595 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":84,"threads_started":1}
I20260812 01:21:50.519029 17864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:50.518707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:38787 (local address 127.24.0.129:56727)
0812 01:21:50.518920 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:50.518965 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":77,"threads_started":1}
I20260812 01:21:50.529991 17866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:50.529759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:38787 (local address 127.24.0.130:34435)
0812 01:21:50.529910 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:50.529947 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":72,"threads_started":1}
I20260812 01:21:50.660375 17867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:50.660159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:38787 (local address 127.24.0.131:52843)
0812 01:21:50.660284 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:50.660330 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":62,"threads_started":1}
I20260812 01:21:51.263908 17869 raft_consensus.cc:482] T 4f224233417d42b989c83b7df00df3f6 P 41ce5ec82b034993aaa0a6e2617407a7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 01:21:51.264089 17869 raft_consensus.cc:504] T 4f224233417d42b989c83b7df00df3f6 P 41ce5ec82b034993aaa0a6e2617407a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d48ccc189c44aa59f80d12affe2a109" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 43669 } } peers { permanent_uuid: "0748ca23dafe4fc88570234338ffdec1" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 35331 } } peers { permanent_uuid: "41ce5ec82b034993aaa0a6e2617407a7" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 35121 } }
I20260812 01:21:51.264313 17869 leader_election.cc:288] T 4f224233417d42b989c83b7df00df3f6 P 41ce5ec82b034993aaa0a6e2617407a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0d48ccc189c44aa59f80d12affe2a109 (127.24.0.131:43669), 0748ca23dafe4fc88570234338ffdec1 (127.24.0.130:35331)
W20260812 01:21:51.264839 17148 leader_election.cc:341] T 4f224233417d42b989c83b7df00df3f6 P 41ce5ec82b034993aaa0a6e2617407a7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0d48ccc189c44aa59f80d12affe2a109 (127.24.0.131:43669): Not found: Tablet not found: 4f224233417d42b989c83b7df00df3f6
W20260812 01:21:51.264907 17148 leader_election.cc:341] T 4f224233417d42b989c83b7df00df3f6 P 41ce5ec82b034993aaa0a6e2617407a7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0748ca23dafe4fc88570234338ffdec1 (127.24.0.130:35331): Not found: Tablet not found: 4f224233417d42b989c83b7df00df3f6
I20260812 01:21:51.264957 17148 leader_election.cc:302] T 4f224233417d42b989c83b7df00df3f6 P 41ce5ec82b034993aaa0a6e2617407a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41ce5ec82b034993aaa0a6e2617407a7; no voters: 0748ca23dafe4fc88570234338ffdec1, 0d48ccc189c44aa59f80d12affe2a109
I20260812 01:21:51.265089 17869 raft_consensus.cc:2726] T 4f224233417d42b989c83b7df00df3f6 P 41ce5ec82b034993aaa0a6e2617407a7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 01:21:51.519790 17870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:51.519556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:38787 (local address 127.24.0.129:37717)
0812 01:21:51.519704 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:51.519743 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":80,"threads_started":1}
I20260812 01:21:51.530815 17872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:51.530560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:38787 (local address 127.24.0.130:51017)
0812 01:21:51.530735 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:51.530766 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":69,"threads_started":1}
I20260812 01:21:51.661075 17873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:51.660827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:38787 (local address 127.24.0.131:43203)
0812 01:21:51.660976 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:51.661019 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":65,"threads_started":1}
I20260812 01:21:52.520524 17874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:52.520311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:38787 (local address 127.24.0.129:35187)
0812 01:21:52.520441 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:52.520476 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":71,"threads_started":1}
I20260812 01:21:52.531456 17875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:52.531255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:38787 (local address 127.24.0.130:57539)
0812 01:21:52.531385 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:52.531414 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":79,"threads_started":1}
I20260812 01:21:52.661875 17877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:52.661604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:38787 (local address 127.24.0.131:56685)
0812 01:21:52.661779 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:52.661825 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":87,"threads_started":1}
I20260812 01:21:53.261621 17879 raft_consensus.cc:482] T 4f224233417d42b989c83b7df00df3f6 P 41ce5ec82b034993aaa0a6e2617407a7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 01:21:53.261724 17879 raft_consensus.cc:504] T 4f224233417d42b989c83b7df00df3f6 P 41ce5ec82b034993aaa0a6e2617407a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d48ccc189c44aa59f80d12affe2a109" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 43669 } } peers { permanent_uuid: "0748ca23dafe4fc88570234338ffdec1" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 35331 } } peers { permanent_uuid: "41ce5ec82b034993aaa0a6e2617407a7" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 35121 } }
I20260812 01:21:53.261916 17879 leader_election.cc:288] T 4f224233417d42b989c83b7df00df3f6 P 41ce5ec82b034993aaa0a6e2617407a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0d48ccc189c44aa59f80d12affe2a109 (127.24.0.131:43669), 0748ca23dafe4fc88570234338ffdec1 (127.24.0.130:35331)
W20260812 01:21:53.262411 17148 leader_election.cc:341] T 4f224233417d42b989c83b7df00df3f6 P 41ce5ec82b034993aaa0a6e2617407a7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0748ca23dafe4fc88570234338ffdec1 (127.24.0.130:35331): Not found: Tablet not found: 4f224233417d42b989c83b7df00df3f6
W20260812 01:21:53.262507 17148 leader_election.cc:341] T 4f224233417d42b989c83b7df00df3f6 P 41ce5ec82b034993aaa0a6e2617407a7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0d48ccc189c44aa59f80d12affe2a109 (127.24.0.131:43669): Not found: Tablet not found: 4f224233417d42b989c83b7df00df3f6
I20260812 01:21:53.262542 17148 leader_election.cc:302] T 4f224233417d42b989c83b7df00df3f6 P 41ce5ec82b034993aaa0a6e2617407a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41ce5ec82b034993aaa0a6e2617407a7; no voters: 0748ca23dafe4fc88570234338ffdec1, 0d48ccc189c44aa59f80d12affe2a109
I20260812 01:21:53.262634 17879 raft_consensus.cc:2726] T 4f224233417d42b989c83b7df00df3f6 P 41ce5ec82b034993aaa0a6e2617407a7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 01:21:53.524488 17880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:53.522688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:38787 (local address 127.24.0.129:49523)
0812 01:21:53.524384 (+ 1696us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:53.524425 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1676,"thread_start_us":1641,"threads_started":1}
I20260812 01:21:53.532220 17881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:53.531907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:38787 (local address 127.24.0.130:43141)
0812 01:21:53.532052 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:53.532123 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":70,"threads_started":1}
I20260812 01:21:53.662587 17884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:53.662354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:38787 (local address 127.24.0.131:54771)
0812 01:21:53.662485 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:53.662518 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":67,"threads_started":1}
I20260812 01:21:54.525156 17887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:54.524955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:38787 (local address 127.24.0.129:40757)
0812 01:21:54.525075 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:54.525113 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":64,"threads_started":1}
I20260812 01:21:54.534759 17889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:54.534561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:38787 (local address 127.24.0.130:46461)
0812 01:21:54.534686 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:54.534719 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":64,"threads_started":1}
I20260812 01:21:54.657271 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 17046
I20260812 01:21:54.665982 17890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:54.665694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:38787 (local address 127.24.0.131:40649)
0812 01:21:54.665875 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:54.665920 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":58,"threads_started":1}
I20260812 01:21:54.670647 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 17356
I20260812 01:21:54.685272 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 17590
2026-08-12T01:21:54Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18890 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 01:21:54 dist-test-slave-t3q3 krb5kdc[17894](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 01:21:54 dist-test-slave-t3q3 krb5kdc[17894](info): set up 2 sockets
Aug 12 01:21:54 dist-test-slave-t3q3 krb5kdc[17894](info): commencing operation
krb5kdc: starting...
W20260812 01:21:55.956745 24578 mini_kdc.cc:121] Time spent starting KDC: real 1.247s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/krb5kdc/test-user.keytab.
Aug 12 01:21:56 dist-test-slave-t3q3 krb5kdc[17894](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497716, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T01:21:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:21:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.0.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.0.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.0.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab.
Entry for principal kudu/127.24.0.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab.
I20260812 01:21:56.080124 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.190:37371
--webserver_interface=127.24.0.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab
--principal=kudu/127.24.0.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:40363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.0.190:37371
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:21:56.162595 17916 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:21:56.162829 17916 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:21:56.162868 17916 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:21:56.164145 17916 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260812 01:21:56.164230 17916 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260812 01:21:56.164263 17916 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260812 01:21:56.164289 17916 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:21:56.164335 17916 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:21:56.164366 17916 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:21:56.164395 17916 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:21:56.164425 17916 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:21:56.166061 17916 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:40363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.24.0.190:37371
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab
--principal=kudu/127.24.0.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.190:37371
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.0.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17916
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:21:56.166349 17916 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:21:56.166620 17916 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:21:56.172097 17927 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:21:56.172094 17926 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:21:56.172188 17930 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:21:56.172197 17916 server_base.cc:1034] running on GCE node
I20260812 01:21:56.172689 17916 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:21:56.172998 17916 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:21:56.174139 17916 hybrid_clock.cc:648] HybridClock initialized: now 1786497716174127 us; error 28 us; skew 500 ppm
Aug 12 01:21:56 dist-test-slave-t3q3 krb5kdc[17894](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497716, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:21:56.176779 17916 init.cc:377] Logged in from keytab as kudu/127.24.0.190@KRBTEST.COM (short username kudu)
I20260812 01:21:56.177253 17916 webserver.cc:460] Webserver started at http://127.24.0.190:43471/ using document root <none> and password file <none>
I20260812 01:21:56.177491 17916 fs_manager.cc:359] Metadata directory not provided
I20260812 01:21:56.177556 17916 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:21:56.177704 17916 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:21:56.178771 17916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/master-0/data/instance:
uuid: "30cc471efc684639a2d06046514aaf95"
format_stamp: "Formatted at 2026-08-12 01:21:56 on dist-test-slave-t3q3"
server_key: "6c3c8d0e32baad54521b8a63f0346eae"
server_key_iv: "2784481ffa70883c7439aee8d9d1c5c8"
server_key_version: "encryptionkey@0"
I20260812 01:21:56.179122 17916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/master-0/wal/instance:
uuid: "30cc471efc684639a2d06046514aaf95"
format_stamp: "Formatted at 2026-08-12 01:21:56 on dist-test-slave-t3q3"
server_key: "6c3c8d0e32baad54521b8a63f0346eae"
server_key_iv: "2784481ffa70883c7439aee8d9d1c5c8"
server_key_version: "encryptionkey@0"
I20260812 01:21:56.180608 17916 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 01:21:56.181358 17943 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:21:56.181537 17916 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:21:56.181639 17916 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/master-0/wal
uuid: "30cc471efc684639a2d06046514aaf95"
format_stamp: "Formatted at 2026-08-12 01:21:56 on dist-test-slave-t3q3"
server_key: "6c3c8d0e32baad54521b8a63f0346eae"
server_key_iv: "2784481ffa70883c7439aee8d9d1c5c8"
server_key_version: "encryptionkey@0"
I20260812 01:21:56.181712 17916 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:21:56.200271 17916 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:21:56.202301 17916 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:21:56.202464 17916 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:21:56.206813 17916 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.190:37371
I20260812 01:21:56.206863 18037 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.190:37371 every 8 connection(s)
I20260812 01:21:56.207226 17916 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
I20260812 01:21:56.208112 18038 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:21:56.210316 18038 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30cc471efc684639a2d06046514aaf95: Bootstrap starting.
I20260812 01:21:56.210856 18038 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 30cc471efc684639a2d06046514aaf95: Neither blocks nor log segments found. Creating new log.
I20260812 01:21:56.211130 18038 log.cc:824] T 00000000000000000000000000000000 P 30cc471efc684639a2d06046514aaf95: Log is configured to *not* fsync() on all Append() calls
I20260812 01:21:56.211753 18038 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30cc471efc684639a2d06046514aaf95: No bootstrap required, opened a new log
I20260812 01:21:56.213395 18038 raft_consensus.cc:348] T 00000000000000000000000000000000 P 30cc471efc684639a2d06046514aaf95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30cc471efc684639a2d06046514aaf95" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 37371 } }
I20260812 01:21:56.213536 18038 raft_consensus.cc:374] T 00000000000000000000000000000000 P 30cc471efc684639a2d06046514aaf95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:21:56.213558 18038 raft_consensus.cc:729] T 00000000000000000000000000000000 P 30cc471efc684639a2d06046514aaf95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30cc471efc684639a2d06046514aaf95, State: Initialized, Role: FOLLOWER
I20260812 01:21:56.213688 18038 consensus_queue.cc:260] T 00000000000000000000000000000000 P 30cc471efc684639a2d06046514aaf95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30cc471efc684639a2d06046514aaf95" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 37371 } }
I20260812 01:21:56.213768 18038 raft_consensus.cc:388] T 00000000000000000000000000000000 P 30cc471efc684639a2d06046514aaf95 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:21:56.213790 18038 raft_consensus.cc:482] T 00000000000000000000000000000000 P 30cc471efc684639a2d06046514aaf95 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:21:56.213819 18038 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 30cc471efc684639a2d06046514aaf95 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:21:56.214313 18038 raft_consensus.cc:504] T 00000000000000000000000000000000 P 30cc471efc684639a2d06046514aaf95 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30cc471efc684639a2d06046514aaf95" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 37371 } }
I20260812 01:21:56.214396 18038 leader_election.cc:302] T 00000000000000000000000000000000 P 30cc471efc684639a2d06046514aaf95 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 30cc471efc684639a2d06046514aaf95; no voters:
I20260812 01:21:56.214543 18038 leader_election.cc:288] T 00000000000000000000000000000000 P 30cc471efc684639a2d06046514aaf95 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 01:21:56.214615 18042 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 30cc471efc684639a2d06046514aaf95 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:21:56.214778 18042 raft_consensus.cc:686] T 00000000000000000000000000000000 P 30cc471efc684639a2d06046514aaf95 [term 1 LEADER]: Becoming Leader. State: Replica: 30cc471efc684639a2d06046514aaf95, State: Running, Role: LEADER
I20260812 01:21:56.214809 18038 sys_catalog.cc:564] T 00000000000000000000000000000000 P 30cc471efc684639a2d06046514aaf95 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:21:56.214866 18042 consensus_queue.cc:237] T 00000000000000000000000000000000 P 30cc471efc684639a2d06046514aaf95 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30cc471efc684639a2d06046514aaf95" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 37371 } }
I20260812 01:21:56.215458 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 17916
I20260812 01:21:56.215572 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/master-0/wal/instance
I20260812 01:21:56.215598 18044 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30cc471efc684639a2d06046514aaf95 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "30cc471efc684639a2d06046514aaf95" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30cc471efc684639a2d06046514aaf95" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 37371 } } }
I20260812 01:21:56.215613 18045 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30cc471efc684639a2d06046514aaf95 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 30cc471efc684639a2d06046514aaf95. Latest consensus state: current_term: 1 leader_uuid: "30cc471efc684639a2d06046514aaf95" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30cc471efc684639a2d06046514aaf95" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 37371 } } }
I20260812 01:21:56.215689 18044 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30cc471efc684639a2d06046514aaf95 [sys.catalog]: This master's current role is: LEADER
I20260812 01:21:56.215695 18045 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30cc471efc684639a2d06046514aaf95 [sys.catalog]: This master's current role is: LEADER
I20260812 01:21:56.215788 24578 external_mini_cluster.cc:1428] Setting key 4616a7241890877e7831a049da1e4484
I20260812 01:21:56.215945 18056 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:21:56.216961 18056 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:21:56.218672 18056 catalog_manager.cc:1347] Generated new cluster ID: d4417e3d8d03430ab68a6c014449d70a
I20260812 01:21:56.218779 18056 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 01:21:56 dist-test-slave-t3q3 krb5kdc[17894](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497716, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
I20260812 01:21:56.227757 18071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:56.216444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40264 (local address 127.24.0.190:37371)
0812 01:21:56.216638 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0812 01:21:56.216647 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 01:21:56.216680 (+ 33us) server_negotiation.cc:407] Connection header received
0812 01:21:56.216818 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:21:56.216826 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:21:56.216886 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:21:56.217030 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:21:56.217848 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:56.219142 (+ 1294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:56.220297 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:56.220970 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:56.221431 (+ 461us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:56.224234 (+ 2803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:21:56.224238 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:21:56.224239 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:21:56.224250 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:21:56.226180 (+ 1930us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:21:56.226571 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:21:56.226574 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:21:56.226574 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:21:56.226658 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:21:56.226879 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:21:56.226880 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:21:56.226880 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:21:56.227129 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:21:56.227285 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0812 01:21:56.227388 (+ 103us) server_negotiation.cc:299] Negotiation successful
0812 01:21:56.227536 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":85,"threads_started":1}
I20260812 01:21:56.238922 18056 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:21:56.239419 18056 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:21:56.250417 18056 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 30cc471efc684639a2d06046514aaf95: Generated new TSK 0
I20260812 01:21:56.250603 18056 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.0.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.0.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.0.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.129.keytab.
Entry for principal kudu/127.24.0.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.129.keytab.
I20260812 01:21:56.280968 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.0.129:0
--local_ip_for_outbound_sockets=127.24.0.129
--webserver_interface=127.24.0.129
--webserver_port=0
--tserver_master_addrs=127.24.0.190:37371
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.129.keytab
--principal=kudu/127.24.0.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:40363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:21:56.372181 18080 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:21:56.372397 18080 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:21:56.372424 18080 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:21:56.373716 18080 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:21:56.373764 18080 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:21:56.373828 18080 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.0.129
I20260812 01:21:56.375439 18080 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:40363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.129.keytab
--principal=kudu/127.24.0.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.0.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.0.129
--webserver_port=0
--tserver_master_addrs=127.24.0.190:37371
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18080
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.0.129
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:21:56.375706 18080 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:21:56.375952 18080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:21:56.381487 18097 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:21:56.381497 18080 server_base.cc:1034] running on GCE node
W20260812 01:21:56.381508 18089 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:21:56.381529 18090 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:21:56.381908 18080 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:21:56.382160 18080 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:21:56.383325 18080 hybrid_clock.cc:648] HybridClock initialized: now 1786497716383312 us; error 43 us; skew 500 ppm
Aug 12 01:21:56 dist-test-slave-t3q3 krb5kdc[17894](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497716, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:21:56.386108 18080 init.cc:377] Logged in from keytab as kudu/127.24.0.129@KRBTEST.COM (short username kudu)
I20260812 01:21:56.386543 18080 webserver.cc:460] Webserver started at http://127.24.0.129:36545/ using document root <none> and password file <none>
I20260812 01:21:56.386781 18080 fs_manager.cc:359] Metadata directory not provided
I20260812 01:21:56.386847 18080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:21:56.386996 18080 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:21:56.388127 18080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-0/data/instance:
uuid: "55139ee42d004825bbc322758eca0733"
format_stamp: "Formatted at 2026-08-12 01:21:56 on dist-test-slave-t3q3"
server_key: "e505175fce3d46c439ffb1acc925b3aa"
server_key_iv: "5824cdee799dda6087b788590ad2a7ad"
server_key_version: "encryptionkey@0"
I20260812 01:21:56.388510 18080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-0/wal/instance:
uuid: "55139ee42d004825bbc322758eca0733"
format_stamp: "Formatted at 2026-08-12 01:21:56 on dist-test-slave-t3q3"
server_key: "e505175fce3d46c439ffb1acc925b3aa"
server_key_iv: "5824cdee799dda6087b788590ad2a7ad"
server_key_version: "encryptionkey@0"
I20260812 01:21:56.389869 18080 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 01:21:56.390630 18110 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:21:56.390798 18080 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 01:21:56.390899 18080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-0/wal
uuid: "55139ee42d004825bbc322758eca0733"
format_stamp: "Formatted at 2026-08-12 01:21:56 on dist-test-slave-t3q3"
server_key: "e505175fce3d46c439ffb1acc925b3aa"
server_key_iv: "5824cdee799dda6087b788590ad2a7ad"
server_key_version: "encryptionkey@0"
I20260812 01:21:56.390980 18080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:21:56.413879 18080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:21:56.416095 18080 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:21:56.416263 18080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:21:56.416550 18080 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:21:56.416923 18080 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:21:56.416977 18080 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:21:56.417021 18080 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:21:56.417060 18080 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:21:56.422933 18080 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.129:41109
I20260812 01:21:56.422984 18304 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.129:41109 every 8 connection(s)
I20260812 01:21:56.423314 18080 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-0/data/info.pb
I20260812 01:21:56.427688 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 18080
I20260812 01:21:56.427815 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-0/wal/instance
I20260812 01:21:56.428027 24578 external_mini_cluster.cc:1428] Setting key cf2f3d75e4176cee13d59b86e30f9980
Aug 12 01:21:56 dist-test-slave-t3q3 krb5kdc[17894](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497716, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.129@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
I20260812 01:21:56.433394 18307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:56.424032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:37371 (local address 127.24.0.129:34927)
0812 01:21:56.424236 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:56.424273 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 01:21:56.424788 (+ 515us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:21:56.425050 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:21:56.425053 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:21:56.425668 (+ 615us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:21:56.425866 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:21:56.425871 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:21:56.426587 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:21:56.426589 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:21:56.427391 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:21:56.427393 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:21:56.427817 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:21:56.427818 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:21:56.427947 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:56.428563 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:21:56.428572 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:21:56.430011 (+ 1439us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:21:56.431969 (+ 1958us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:21:56.431976 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:21:56.431978 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:21:56.432276 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:21:56.432491 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:21:56.432491 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:21:56.432492 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:21:56.432609 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:21:56.432897 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:21:56.432901 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:21:56.433029 (+ 128us) client_negotiation.cc:769] Sending connection context
0812 01:21:56.433109 (+ 80us) client_negotiation.cc:240] Negotiation successful
0812 01:21:56.433231 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":118,"threads_started":1}
I20260812 01:21:56.433437 18071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:56.424050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.129:34927 (local address 127.24.0.190:37371)
0812 01:21:56.424149 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0812 01:21:56.424151 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:21:56.424363 (+ 212us) server_negotiation.cc:407] Connection header received
0812 01:21:56.424902 (+ 539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:21:56.424904 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:21:56.424920 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:21:56.424975 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:21:56.425968 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:56.426490 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:56.427503 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:56.427729 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:56.427807 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:56.430121 (+ 2314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:21:56.430122 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:21:56.430123 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:21:56.430138 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:21:56.431880 (+ 1742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:21:56.432346 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:21:56.432347 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:21:56.432348 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:21:56.432417 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:21:56.432679 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:21:56.432680 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:21:56.432680 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:21:56.432836 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:21:56.432905 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 01:21:56.433170 (+ 265us) server_negotiation.cc:299] Negotiation successful
0812 01:21:56.433289 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260812 01:21:56.434154 18305 heartbeater.cc:347] Connected to a master server at 127.24.0.190:37371
I20260812 01:21:56.434266 18305 heartbeater.cc:464] Registering TS with master...
I20260812 01:21:56.434455 18305 heartbeater.cc:511] Master 127.24.0.190:37371 requested a full tablet report, sending...
I20260812 01:21:56.435143 17969 ts_manager.cc:194] Registered new tserver with Master: 55139ee42d004825bbc322758eca0733 (127.24.0.129:41109)
I20260812 01:21:56.436182 17969 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.0.129@KRBTEST.COM'} at 127.24.0.129:34927
WARNING: no policy specified for kudu/127.24.0.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.0.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.0.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.130.keytab.
Entry for principal kudu/127.24.0.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.130.keytab.
I20260812 01:21:56.454839 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.0.130:0
--local_ip_for_outbound_sockets=127.24.0.130
--webserver_interface=127.24.0.130
--webserver_port=0
--tserver_master_addrs=127.24.0.190:37371
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.130.keytab
--principal=kudu/127.24.0.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:40363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:21:56.538769 18311 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:21:56.538988 18311 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:21:56.539013 18311 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:21:56.540199 18311 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:21:56.540242 18311 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:21:56.540269 18311 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.0.130
I20260812 01:21:56.541757 18311 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:40363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.130.keytab
--principal=kudu/127.24.0.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.0.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.0.130
--webserver_port=0
--tserver_master_addrs=127.24.0.190:37371
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18311
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.0.130
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:21:56.542016 18311 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:21:56.542268 18311 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:21:56.547434 18324 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:21:56.547430 18321 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:21:56.547433 18326 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:21:56.547715 18311 server_base.cc:1034] running on GCE node
I20260812 01:21:56.547897 18311 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:21:56.548194 18311 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:21:56.549381 18311 hybrid_clock.cc:648] HybridClock initialized: now 1786497716549361 us; error 30 us; skew 500 ppm
Aug 12 01:21:56 dist-test-slave-t3q3 krb5kdc[17894](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497716, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:21:56.551976 18311 init.cc:377] Logged in from keytab as kudu/127.24.0.130@KRBTEST.COM (short username kudu)
I20260812 01:21:56.552456 18311 webserver.cc:460] Webserver started at http://127.24.0.130:41971/ using document root <none> and password file <none>
I20260812 01:21:56.552690 18311 fs_manager.cc:359] Metadata directory not provided
I20260812 01:21:56.552736 18311 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:21:56.552896 18311 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:21:56.553960 18311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-1/data/instance:
uuid: "6cf4f241cc73458ea35718b385f9207f"
format_stamp: "Formatted at 2026-08-12 01:21:56 on dist-test-slave-t3q3"
server_key: "7961aa150737e2fcea0b71ba23418d1e"
server_key_iv: "6d79c2a710d80a3af872ef54a5d36a45"
server_key_version: "encryptionkey@0"
I20260812 01:21:56.554335 18311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-1/wal/instance:
uuid: "6cf4f241cc73458ea35718b385f9207f"
format_stamp: "Formatted at 2026-08-12 01:21:56 on dist-test-slave-t3q3"
server_key: "7961aa150737e2fcea0b71ba23418d1e"
server_key_iv: "6d79c2a710d80a3af872ef54a5d36a45"
server_key_version: "encryptionkey@0"
I20260812 01:21:56.555677 18311 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 01:21:56.556478 18335 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:21:56.556648 18311 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 01:21:56.556738 18311 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-1/wal
uuid: "6cf4f241cc73458ea35718b385f9207f"
format_stamp: "Formatted at 2026-08-12 01:21:56 on dist-test-slave-t3q3"
server_key: "7961aa150737e2fcea0b71ba23418d1e"
server_key_iv: "6d79c2a710d80a3af872ef54a5d36a45"
server_key_version: "encryptionkey@0"
I20260812 01:21:56.556809 18311 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:21:56.565131 18311 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:21:56.567428 18311 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:21:56.567589 18311 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:21:56.567813 18311 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:21:56.568158 18311 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:21:56.568212 18311 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:21:56.568252 18311 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:21:56.568279 18311 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:21:56.574210 18311 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.130:39345
I20260812 01:21:56.574283 18543 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.130:39345 every 8 connection(s)
I20260812 01:21:56.574615 18311 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-1/data/info.pb
I20260812 01:21:56.579963 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 18311
I20260812 01:21:56.580135 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-1/wal/instance
I20260812 01:21:56.580367 24578 external_mini_cluster.cc:1428] Setting key 534b803f2d1dc8d6c0215b90096ba734
Aug 12 01:21:56 dist-test-slave-t3q3 krb5kdc[17894](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497716, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.130@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
I20260812 01:21:56.584039 18071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:56.575342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.130:37807 (local address 127.24.0.190:37371)
0812 01:21:56.575410 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0812 01:21:56.575412 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:21:56.575595 (+ 183us) server_negotiation.cc:407] Connection header received
0812 01:21:56.576104 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:21:56.576105 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:21:56.576117 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:21:56.576188 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:21:56.577127 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:56.577645 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:56.578649 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:56.578835 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:56.578890 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:56.581247 (+ 2357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:21:56.581249 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:21:56.581250 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:21:56.581257 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:21:56.582581 (+ 1324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:21:56.582997 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:21:56.582999 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:21:56.582999 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:21:56.583061 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:21:56.583322 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:21:56.583323 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:21:56.583323 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:21:56.583449 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:21:56.583530 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 01:21:56.583852 (+ 322us) server_negotiation.cc:299] Negotiation successful
0812 01:21:56.583947 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 01:21:56.584268 18548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:56.575292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:37371 (local address 127.24.0.130:37807)
0812 01:21:56.575461 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:56.575504 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 01:21:56.575981 (+ 477us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:21:56.576242 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:21:56.576245 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:21:56.576812 (+ 567us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:21:56.577010 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:21:56.577015 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:21:56.577747 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:21:56.577748 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:21:56.578573 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:21:56.578576 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:21:56.578907 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:21:56.578908 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:21:56.579005 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:56.579484 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:21:56.579492 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:21:56.581125 (+ 1633us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:21:56.582679 (+ 1554us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:21:56.582688 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:21:56.582690 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:21:56.582919 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:21:56.583139 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:21:56.583140 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:21:56.583140 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:21:56.583257 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:21:56.583519 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:21:56.583527 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:21:56.583704 (+ 177us) client_negotiation.cc:769] Sending connection context
0812 01:21:56.583829 (+ 125us) client_negotiation.cc:240] Negotiation successful
0812 01:21:56.583995 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":94,"threads_started":1}
I20260812 01:21:56.584748 18545 heartbeater.cc:347] Connected to a master server at 127.24.0.190:37371
I20260812 01:21:56.584856 18545 heartbeater.cc:464] Registering TS with master...
I20260812 01:21:56.585040 18545 heartbeater.cc:511] Master 127.24.0.190:37371 requested a full tablet report, sending...
I20260812 01:21:56.585501 17969 ts_manager.cc:194] Registered new tserver with Master: 6cf4f241cc73458ea35718b385f9207f (127.24.0.130:39345)
I20260812 01:21:56.585927 17969 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.0.130@KRBTEST.COM'} at 127.24.0.130:37807
WARNING: no policy specified for kudu/127.24.0.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.0.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.0.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.131.keytab.
Entry for principal kudu/127.24.0.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.131.keytab.
I20260812 01:21:56.607003 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.0.131:0
--local_ip_for_outbound_sockets=127.24.0.131
--webserver_interface=127.24.0.131
--webserver_port=0
--tserver_master_addrs=127.24.0.190:37371
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.131.keytab
--principal=kudu/127.24.0.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:40363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:21:56.689777 18553 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:21:56.689965 18553 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:21:56.689987 18553 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:21:56.691035 18553 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:21:56.691069 18553 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:21:56.691095 18553 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.0.131
I20260812 01:21:56.692483 18553 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:40363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.131.keytab
--principal=kudu/127.24.0.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.0.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.0.131
--webserver_port=0
--tserver_master_addrs=127.24.0.190:37371
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18553
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.0.131
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:21:56.692680 18553 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:21:56.692875 18553 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:21:56.697875 18569 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:21:56.697901 18562 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:21:56.697875 18561 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:21:56.697889 18553 server_base.cc:1034] running on GCE node
I20260812 01:21:56.698254 18553 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:21:56.698539 18553 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:21:56.699704 18553 hybrid_clock.cc:648] HybridClock initialized: now 1786497716699693 us; error 36 us; skew 500 ppm
Aug 12 01:21:56 dist-test-slave-t3q3 krb5kdc[17894](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497716, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:21:56.702286 18553 init.cc:377] Logged in from keytab as kudu/127.24.0.131@KRBTEST.COM (short username kudu)
I20260812 01:21:56.702729 18553 webserver.cc:460] Webserver started at http://127.24.0.131:33579/ using document root <none> and password file <none>
I20260812 01:21:56.702937 18553 fs_manager.cc:359] Metadata directory not provided
I20260812 01:21:56.703001 18553 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:21:56.703150 18553 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:21:56.704217 18553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-2/data/instance:
uuid: "2fb3a91da6ad4a2390d98ba13a53ed06"
format_stamp: "Formatted at 2026-08-12 01:21:56 on dist-test-slave-t3q3"
server_key: "2984abd5ee12c68ca949e6f9be6adc19"
server_key_iv: "4acc62bdf1abc43efab71c1f259d601a"
server_key_version: "encryptionkey@0"
I20260812 01:21:56.704582 18553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-2/wal/instance:
uuid: "2fb3a91da6ad4a2390d98ba13a53ed06"
format_stamp: "Formatted at 2026-08-12 01:21:56 on dist-test-slave-t3q3"
server_key: "2984abd5ee12c68ca949e6f9be6adc19"
server_key_iv: "4acc62bdf1abc43efab71c1f259d601a"
server_key_version: "encryptionkey@0"
I20260812 01:21:56.705927 18553 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 01:21:56.706642 18579 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:21:56.706818 18553 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 01:21:56.706919 18553 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-2/wal
uuid: "2fb3a91da6ad4a2390d98ba13a53ed06"
format_stamp: "Formatted at 2026-08-12 01:21:56 on dist-test-slave-t3q3"
server_key: "2984abd5ee12c68ca949e6f9be6adc19"
server_key_iv: "4acc62bdf1abc43efab71c1f259d601a"
server_key_version: "encryptionkey@0"
I20260812 01:21:56.706987 18553 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:21:56.727837 18553 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:21:56.730718 18553 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:21:56.730881 18553 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:21:56.731168 18553 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:21:56.731549 18553 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:21:56.731599 18553 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:21:56.731643 18553 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:21:56.731673 18553 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:21:56.739279 18553 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.131:35249
I20260812 01:21:56.739348 18767 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.131:35249 every 8 connection(s)
I20260812 01:21:56.739667 18553 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-2/data/info.pb
I20260812 01:21:56.742345 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 18553
I20260812 01:21:56.742421 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-2/wal/instance
I20260812 01:21:56.742604 24578 external_mini_cluster.cc:1428] Setting key 03ae81ffc438eca68363ccd39440f633
Aug 12 01:21:56 dist-test-slave-t3q3 krb5kdc[17894](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497716, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.131@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
I20260812 01:21:56.749102 18071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:56.740411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.131:40877 (local address 127.24.0.190:37371)
0812 01:21:56.740457 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 01:21:56.740460 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:21:56.740651 (+ 191us) server_negotiation.cc:407] Connection header received
0812 01:21:56.741128 (+ 477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:21:56.741129 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:21:56.741139 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:21:56.741190 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:21:56.742118 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:56.742720 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:56.744275 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:56.744464 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:56.744531 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:56.746656 (+ 2125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:21:56.746657 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:21:56.746657 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:21:56.746663 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:21:56.747882 (+ 1219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:21:56.748336 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:21:56.748336 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:21:56.748337 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:21:56.748391 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:21:56.748606 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:21:56.748607 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:21:56.748607 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:21:56.748705 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:21:56.748753 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0812 01:21:56.748972 (+ 219us) server_negotiation.cc:299] Negotiation successful
0812 01:21:56.749034 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 01:21:56.749577 18776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:56.740384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:37371 (local address 127.24.0.131:40877)
0812 01:21:56.740531 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:56.740566 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0812 01:21:56.741023 (+ 457us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:21:56.741251 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:21:56.741255 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:21:56.741820 (+ 565us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:21:56.742015 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:21:56.742019 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:21:56.742802 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:21:56.742803 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:21:56.744176 (+ 1373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:21:56.744179 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:21:56.744551 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:21:56.744553 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:21:56.744654 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:56.745179 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:21:56.745187 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:21:56.746562 (+ 1375us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:21:56.747960 (+ 1398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:21:56.747966 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:21:56.747968 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:21:56.748268 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:21:56.748453 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:21:56.748454 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:21:56.748454 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:21:56.748557 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:21:56.748751 (+ 194us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:21:56.748754 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:21:56.748863 (+ 109us) client_negotiation.cc:769] Sending connection context
0812 01:21:56.749101 (+ 238us) client_negotiation.cc:240] Negotiation successful
0812 01:21:56.749306 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":82,"threads_started":1}
I20260812 01:21:56.750056 18768 heartbeater.cc:347] Connected to a master server at 127.24.0.190:37371
I20260812 01:21:56.750147 18768 heartbeater.cc:464] Registering TS with master...
I20260812 01:21:56.750371 18768 heartbeater.cc:511] Master 127.24.0.190:37371 requested a full tablet report, sending...
I20260812 01:21:56.750802 17969 ts_manager.cc:194] Registered new tserver with Master: 2fb3a91da6ad4a2390d98ba13a53ed06 (127.24.0.131:35249)
I20260812 01:21:56.751164 17969 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.0.131@KRBTEST.COM'} at 127.24.0.131:40877
I20260812 01:21:56.754329 24578 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 01:21:56.762039 18071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:56.755664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40270 (local address 127.24.0.190:37371)
0812 01:21:56.755744 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0812 01:21:56.755746 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:21:56.755866 (+ 120us) server_negotiation.cc:407] Connection header received
0812 01:21:56.755945 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:21:56.755946 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:21:56.755956 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:21:56.756055 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:21:56.756640 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:56.757121 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:56.757869 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:56.758066 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:56.758141 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:56.759354 (+ 1213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:21:56.759356 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:21:56.759356 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:21:56.759362 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:21:56.760889 (+ 1527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:21:56.761230 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:21:56.761232 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:21:56.761232 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:21:56.761290 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:21:56.761483 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:21:56.761484 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:21:56.761484 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:21:56.761633 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:21:56.761740 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0812 01:21:56.761851 (+ 111us) server_negotiation.cc:299] Negotiation successful
0812 01:21:56.761932 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 01:21:56.763712 17969 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40270:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 01:21:56.765434 17969 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 01:21:56.771916 18788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:56.767810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.131:35249 (local address 127.0.0.1:44324)
0812 01:21:56.768440 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:56.768451 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 01:21:56.768502 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:21:56.768793 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:21:56.768794 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:21:56.768797 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:21:56.768852 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:21:56.768854 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:21:56.769773 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:21:56.769774 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:21:56.770912 (+ 1138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:21:56.770915 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:21:56.771570 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:21:56.771571 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:21:56.771675 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:56.771690 (+ 15us) client_negotiation.cc:769] Sending connection context
0812 01:21:56.771786 (+ 96us) client_negotiation.cc:240] Negotiation successful
0812 01:21:56.771843 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":616,"mutex_wait_us":14,"thread_start_us":102,"threads_started":1}
I20260812 01:21:56.771927 18803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:56.768231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44324 (local address 127.24.0.131:35249)
0812 01:21:56.768451 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0812 01:21:56.768456 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 01:21:56.768560 (+ 104us) server_negotiation.cc:407] Connection header received
0812 01:21:56.768603 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:21:56.768605 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:21:56.768620 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:21:56.768706 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:21:56.768928 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:56.769670 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:56.771002 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:56.771474 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:56.771570 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:56.771637 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 01:21:56.771806 (+ 169us) server_negotiation.cc:299] Negotiation successful
0812 01:21:56.771848 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":107,"threads_started":1}
I20260812 01:21:56.772596 18665 tablet_service.cc:1467] Processing CreateTablet for tablet 1c6f1fb89dde44aaa219c8fb0fb7f32c (DEFAULT_TABLE table=test-table [id=daab6165e8f54d67a06e4b1633f4ed53]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:21:56.772771 18795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:56.767810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.130:39345 (local address 127.0.0.1:57052)
0812 01:21:56.767968 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:56.767997 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0812 01:21:56.768085 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:21:56.768531 (+ 446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:21:56.768535 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:21:56.768538 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:21:56.768607 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:21:56.768608 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:21:56.770298 (+ 1690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:21:56.770299 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:21:56.771584 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:21:56.771587 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:21:56.772559 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:21:56.772560 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:21:56.772625 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:56.772629 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 01:21:56.772661 (+ 32us) client_negotiation.cc:240] Negotiation successful
0812 01:21:56.772699 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"mutex_wait_us":136,"thread_start_us":45,"threads_started":1}
I20260812 01:21:56.773018 18665 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c6f1fb89dde44aaa219c8fb0fb7f32c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:21:56.773192 18798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:56.768151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57052 (local address 127.24.0.130:39345)
0812 01:21:56.768340 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0812 01:21:56.768344 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 01:21:56.768356 (+ 12us) server_negotiation.cc:407] Connection header received
0812 01:21:56.768400 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:21:56.768402 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:21:56.768419 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:21:56.768508 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:21:56.769019 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:56.770141 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:56.771688 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:56.772453 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:56.772942 (+ 489us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:56.773030 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0812 01:21:56.773056 (+ 26us) server_negotiation.cc:299] Negotiation successful
0812 01:21:56.773102 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":87,"threads_started":1}
I20260812 01:21:56.773285 18793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:56.767807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.129:41109 (local address 127.0.0.1:58182)
0812 01:21:56.767954 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:56.767997 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 01:21:56.768085 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:21:56.768926 (+ 841us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:21:56.768927 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:21:56.768930 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:21:56.768974 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:21:56.768975 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:21:56.770308 (+ 1333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:21:56.770308 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:21:56.771585 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:21:56.771587 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:21:56.773061 (+ 1474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:21:56.773062 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:21:56.773128 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:56.773132 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 01:21:56.773184 (+ 52us) client_negotiation.cc:240] Negotiation successful
0812 01:21:56.773223 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"mutex_wait_us":73,"thread_start_us":94,"threads_started":1}
I20260812 01:21:56.773633 18800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:56.768130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58182 (local address 127.24.0.129:41109)
0812 01:21:56.768753 (+ 623us) server_negotiation.cc:206] Beginning negotiation
0812 01:21:56.768758 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 01:21:56.768768 (+ 10us) server_negotiation.cc:407] Connection header received
0812 01:21:56.768806 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:21:56.768808 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:21:56.768822 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:21:56.768908 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:21:56.769088 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:56.770184 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:56.771696 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:56.772752 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:56.773060 (+ 308us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:56.773168 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0812 01:21:56.773199 (+ 31us) server_negotiation.cc:299] Negotiation successful
0812 01:21:56.773552 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":586,"thread_start_us":103,"threads_started":1}
I20260812 01:21:56.773862 18434 tablet_service.cc:1467] Processing CreateTablet for tablet 1c6f1fb89dde44aaa219c8fb0fb7f32c (DEFAULT_TABLE table=test-table [id=daab6165e8f54d67a06e4b1633f4ed53]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:21:56.774209 18434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c6f1fb89dde44aaa219c8fb0fb7f32c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:21:56.774255 18194 tablet_service.cc:1467] Processing CreateTablet for tablet 1c6f1fb89dde44aaa219c8fb0fb7f32c (DEFAULT_TABLE table=test-table [id=daab6165e8f54d67a06e4b1633f4ed53]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:21:56.774598 18194 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c6f1fb89dde44aaa219c8fb0fb7f32c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:21:56.776584 18805 tablet_bootstrap.cc:492] T 1c6f1fb89dde44aaa219c8fb0fb7f32c P 2fb3a91da6ad4a2390d98ba13a53ed06: Bootstrap starting.
I20260812 01:21:56.777364 18805 tablet_bootstrap.cc:654] T 1c6f1fb89dde44aaa219c8fb0fb7f32c P 2fb3a91da6ad4a2390d98ba13a53ed06: Neither blocks nor log segments found. Creating new log.
I20260812 01:21:56.777474 18806 tablet_bootstrap.cc:492] T 1c6f1fb89dde44aaa219c8fb0fb7f32c P 55139ee42d004825bbc322758eca0733: Bootstrap starting.
I20260812 01:21:56.778054 18805 log.cc:824] T 1c6f1fb89dde44aaa219c8fb0fb7f32c P 2fb3a91da6ad4a2390d98ba13a53ed06: Log is configured to *not* fsync() on all Append() calls
I20260812 01:21:56.778106 18806 tablet_bootstrap.cc:654] T 1c6f1fb89dde44aaa219c8fb0fb7f32c P 55139ee42d004825bbc322758eca0733: Neither blocks nor log segments found. Creating new log.
I20260812 01:21:56.778574 18806 log.cc:824] T 1c6f1fb89dde44aaa219c8fb0fb7f32c P 55139ee42d004825bbc322758eca0733: Log is configured to *not* fsync() on all Append() calls
I20260812 01:21:56.778815 18805 tablet_bootstrap.cc:492] T 1c6f1fb89dde44aaa219c8fb0fb7f32c P 2fb3a91da6ad4a2390d98ba13a53ed06: No bootstrap required, opened a new log
I20260812 01:21:56.778893 18805 ts_tablet_manager.cc:1397] T 1c6f1fb89dde44aaa219c8fb0fb7f32c P 2fb3a91da6ad4a2390d98ba13a53ed06: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 01:21:56.778925 18807 tablet_bootstrap.cc:492] T 1c6f1fb89dde44aaa219c8fb0fb7f32c P 6cf4f241cc73458ea35718b385f9207f: Bootstrap starting.
I20260812 01:21:56.779337 18806 tablet_bootstrap.cc:492] T 1c6f1fb89dde44aaa219c8fb0fb7f32c P 55139ee42d004825bbc322758eca0733: No bootstrap required, opened a new log
I20260812 01:21:56.779410 18806 ts_tablet_manager.cc:1397] T 1c6f1fb89dde44aaa219c8fb0fb7f32c P 55139ee42d004825bbc322758eca0733: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 01:21:56.779593 18807 tablet_bootstrap.cc:654] T 1c6f1fb89dde44aaa219c8fb0fb7f32c P 6cf4f241cc73458ea35718b385f9207f: Neither blocks nor log segments found. Creating new log.
I20260812 01:21:56.780202 18807 log.cc:824] T 1c6f1fb89dde44aaa219c8fb0fb7f32c P 6cf4f241cc73458ea35718b385f9207f: Log is configured to *not* fsync() on all Append() calls
I20260812 01:21:56.781039 18807 tablet_bootstrap.cc:492] T 1c6f1fb89dde44aaa219c8fb0fb7f32c P 6cf4f241cc73458ea35718b385f9207f: No bootstrap required, opened a new log
I20260812 01:21:56.780970 18805 raft_consensus.cc:348] T 1c6f1fb89dde44aaa219c8fb0fb7f32c P 2fb3a91da6ad4a2390d98ba13a53ed06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55139ee42d004825bbc322758eca0733" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 41109 } } peers { permanent_uuid: "2fb3a91da6ad4a2390d98ba13a53ed06" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 35249 } } peers { permanent_uuid: "6cf4f241cc73458ea35718b385f9207f" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39345 } }
I20260812 01:21:56.781119 18807 ts_tablet_manager.cc:1397] T 1c6f1fb89dde44aaa219c8fb0fb7f32c P 6cf4f241cc73458ea35718b385f9207f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 01:21:56.781126 18805 raft_consensus.cc:374] T 1c6f1fb89dde44aaa219c8fb0fb7f32c P 2fb3a91da6ad4a2390d98ba13a53ed06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:21:56.781167 18805 raft_consensus.cc:729] T 1c6f1fb89dde44aaa219c8fb0fb7f32c P 2fb3a91da6ad4a2390d98ba13a53ed06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fb3a91da6ad4a2390d98ba13a53ed06, State: Initialized, Role: FOLLOWER
I20260812 01:21:56.781313 18805 consensus_queue.cc:260] T 1c6f1fb89dde44aaa219c8fb0fb7f32c P 2fb3a91da6ad4a2390d98ba13a53ed06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55139ee42d004825bbc322758eca0733" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 41109 } } peers { permanent_uuid: "2fb3a91da6ad4a2390d98ba13a53ed06" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 35249 } } peers { permanent_uuid: "6cf4f241cc73458ea35718b385f9207f" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39345 } }
I20260812 01:21:56.781586 18805 ts_tablet_manager.cc:1428] T 1c6f1fb89dde44aaa219c8fb0fb7f32c P 2fb3a91da6ad4a2390d98ba13a53ed06: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 01:21:56.781554 18806 raft_consensus.cc:348] T 1c6f1fb89dde44aaa219c8fb0fb7f32c P 55139ee42d004825bbc322758eca0733 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55139ee42d004825bbc322758eca0733" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 41109 } } peers { permanent_uuid: "2fb3a91da6ad4a2390d98ba13a53ed06" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 35249 } } peers { permanent_uuid: "6cf4f241cc73458ea35718b385f9207f" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39345 } }
I20260812 01:21:56.781719 18806 raft_consensus.cc:374] T 1c6f1fb89dde44aaa219c8fb0fb7f32c P 55139ee42d004825bbc322758eca0733 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:21:56.781733 18768 heartbeater.cc:503] Master 127.24.0.190:37371 was elected leader, sending a full tablet report...
I20260812 01:21:56.781754 18806 raft_consensus.cc:729] T 1c6f1fb89dde44aaa219c8fb0fb7f32c P 55139ee42d004825bbc322758eca0733 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55139ee42d004825bbc322758eca0733, State: Initialized, Role: FOLLOWER
I20260812 01:21:56.781884 18806 consensus_queue.cc:260] T 1c6f1fb89dde44aaa219c8fb0fb7f32c P 55139ee42d004825bbc322758eca0733 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55139ee42d004825bbc322758eca0733" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 41109 } } peers { permanent_uuid: "2fb3a91da6ad4a2390d98ba13a53ed06" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 35249 } } peers { permanent_uuid: "6cf4f241cc73458ea35718b385f9207f" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39345 } }
I20260812 01:21:56.782271 18806 ts_tablet_manager.cc:1428] T 1c6f1fb89dde44aaa219c8fb0fb7f32c P 55139ee42d004825bbc322758eca0733: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 01:21:56.782285 18305 heartbeater.cc:503] Master 127.24.0.190:37371 was elected leader, sending a full tablet report...
I20260812 01:21:56.783356 18807 raft_consensus.cc:348] T 1c6f1fb89dde44aaa219c8fb0fb7f32c P 6cf4f241cc73458ea35718b385f9207f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55139ee42d004825bbc322758eca0733" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 41109 } } peers { permanent_uuid: "2fb3a91da6ad4a2390d98ba13a53ed06" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 35249 } } peers { permanent_uuid: "6cf4f241cc73458ea35718b385f9207f" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39345 } }
I20260812 01:21:56.783488 18807 raft_consensus.cc:374] T 1c6f1fb89dde44aaa219c8fb0fb7f32c P 6cf4f241cc73458ea35718b385f9207f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:21:56.783509 18807 raft_consensus.cc:729] T 1c6f1fb89dde44aaa219c8fb0fb7f32c P 6cf4f241cc73458ea35718b385f9207f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cf4f241cc73458ea35718b385f9207f, State: Initialized, Role: FOLLOWER
I20260812 01:21:56.783607 18807 consensus_queue.cc:260] T 1c6f1fb89dde44aaa219c8fb0fb7f32c P 6cf4f241cc73458ea35718b385f9207f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55139ee42d004825bbc322758eca0733" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 41109 } } peers { permanent_uuid: "2fb3a91da6ad4a2390d98ba13a53ed06" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 35249 } } peers { permanent_uuid: "6cf4f241cc73458ea35718b385f9207f" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39345 } }
I20260812 01:21:56.783887 18807 ts_tablet_manager.cc:1428] T 1c6f1fb89dde44aaa219c8fb0fb7f32c P 6cf4f241cc73458ea35718b385f9207f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 01:21:56.783907 18545 heartbeater.cc:503] Master 127.24.0.190:37371 was elected leader, sending a full tablet report...
W20260812 01:21:56.825206 18546 tablet.cc:2369] T 1c6f1fb89dde44aaa219c8fb0fb7f32c P 6cf4f241cc73458ea35718b385f9207f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 01:21:56.924144 18306 tablet.cc:2369] T 1c6f1fb89dde44aaa219c8fb0fb7f32c P 55139ee42d004825bbc322758eca0733: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 01:21:56.947228 18822 raft_consensus.cc:482] T 1c6f1fb89dde44aaa219c8fb0fb7f32c P 6cf4f241cc73458ea35718b385f9207f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 01:21:56.947372 18822 raft_consensus.cc:504] T 1c6f1fb89dde44aaa219c8fb0fb7f32c P 6cf4f241cc73458ea35718b385f9207f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55139ee42d004825bbc322758eca0733" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 41109 } } peers { permanent_uuid: "2fb3a91da6ad4a2390d98ba13a53ed06" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 35249 } } peers { permanent_uuid: "6cf4f241cc73458ea35718b385f9207f" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39345 } }
I20260812 01:21:56.947851 18822 leader_election.cc:288] T 1c6f1fb89dde44aaa219c8fb0fb7f32c P 6cf4f241cc73458ea35718b385f9207f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 55139ee42d004825bbc322758eca0733 (127.24.0.129:41109), 2fb3a91da6ad4a2390d98ba13a53ed06 (127.24.0.131:35249)
I20260812 01:21:56.951692 18800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:56.948165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.130:51151 (local address 127.24.0.129:41109)
0812 01:21:56.948216 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 01:21:56.948219 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:21:56.948259 (+ 40us) server_negotiation.cc:407] Connection header received
0812 01:21:56.948292 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:21:56.948293 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:21:56.948305 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:21:56.948382 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:21:56.948558 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:56.949335 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:56.950774 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:56.951319 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:56.951402 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:56.951412 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 01:21:56.951581 (+ 169us) server_negotiation.cc:299] Negotiation successful
0812 01:21:56.951618 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 01:21:56.951692 18803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:56.948225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.130:53247 (local address 127.24.0.131:35249)
0812 01:21:56.948294 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0812 01:21:56.948297 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:21:56.948367 (+ 70us) server_negotiation.cc:407] Connection header received
0812 01:21:56.948444 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:21:56.948445 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:21:56.948456 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:21:56.948524 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:21:56.948715 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:56.949409 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:56.950774 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:56.951340 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:56.951403 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:56.951412 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 01:21:56.951581 (+ 169us) server_negotiation.cc:299] Negotiation successful
0812 01:21:56.951618 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 01:21:56.951727 18548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:56.948055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.129:41109 (local address 127.24.0.130:51151)
0812 01:21:56.948151 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:56.948164 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 01:21:56.948216 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:21:56.948389 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:21:56.948390 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:21:56.948392 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:21:56.948467 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:21:56.948469 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:21:56.949419 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:21:56.949420 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:21:56.950684 (+ 1264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:21:56.950685 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:21:56.951415 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:21:56.951416 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:21:56.951500 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:56.951503 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 01:21:56.951565 (+ 62us) client_negotiation.cc:240] Negotiation successful
0812 01:21:56.951617 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":13}
I20260812 01:21:56.951731 18823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:56.948137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.131:35249 (local address 127.24.0.130:53247)
0812 01:21:56.948308 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:56.948317 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 01:21:56.948378 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:21:56.948529 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:21:56.948530 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:21:56.948536 (+ 6us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:21:56.948608 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:21:56.948610 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:21:56.949480 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:21:56.949480 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:21:56.950680 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:21:56.950684 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:21:56.951415 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:21:56.951416 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:21:56.951499 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:56.951503 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 01:21:56.951565 (+ 62us) client_negotiation.cc:240] Negotiation successful
0812 01:21:56.951617 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"mutex_wait_us":33,"thread_start_us":113,"threads_started":1}
I20260812 01:21:56.952159 18227 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c6f1fb89dde44aaa219c8fb0fb7f32c" candidate_uuid: "6cf4f241cc73458ea35718b385f9207f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55139ee42d004825bbc322758eca0733" is_pre_election: true
I20260812 01:21:56.952361 18227 raft_consensus.cc:2454] T 1c6f1fb89dde44aaa219c8fb0fb7f32c P 55139ee42d004825bbc322758eca0733 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6cf4f241cc73458ea35718b385f9207f in term 0.
I20260812 01:21:56.952337 18699 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c6f1fb89dde44aaa219c8fb0fb7f32c" candidate_uuid: "6cf4f241cc73458ea35718b385f9207f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fb3a91da6ad4a2390d98ba13a53ed06" is_pre_election: true
I20260812 01:21:56.952494 18699 raft_consensus.cc:2454] T 1c6f1fb89dde44aaa219c8fb0fb7f32c P 2fb3a91da6ad4a2390d98ba13a53ed06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6cf4f241cc73458ea35718b385f9207f in term 0.
I20260812 01:21:56.952602 18336 leader_election.cc:302] T 1c6f1fb89dde44aaa219c8fb0fb7f32c P 6cf4f241cc73458ea35718b385f9207f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55139ee42d004825bbc322758eca0733, 6cf4f241cc73458ea35718b385f9207f; no voters:
I20260812 01:21:56.952741 18822 raft_consensus.cc:2781] T 1c6f1fb89dde44aaa219c8fb0fb7f32c P 6cf4f241cc73458ea35718b385f9207f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 01:21:56.952800 18822 raft_consensus.cc:482] T 1c6f1fb89dde44aaa219c8fb0fb7f32c P 6cf4f241cc73458ea35718b385f9207f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 01:21:56.952824 18822 raft_consensus.cc:3037] T 1c6f1fb89dde44aaa219c8fb0fb7f32c P 6cf4f241cc73458ea35718b385f9207f [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:21:56.953605 18822 raft_consensus.cc:504] T 1c6f1fb89dde44aaa219c8fb0fb7f32c P 6cf4f241cc73458ea35718b385f9207f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55139ee42d004825bbc322758eca0733" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 41109 } } peers { permanent_uuid: "2fb3a91da6ad4a2390d98ba13a53ed06" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 35249 } } peers { permanent_uuid: "6cf4f241cc73458ea35718b385f9207f" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39345 } }
I20260812 01:21:56.953752 18822 leader_election.cc:288] T 1c6f1fb89dde44aaa219c8fb0fb7f32c P 6cf4f241cc73458ea35718b385f9207f [CANDIDATE]: Term 1 election: Requested vote from peers 55139ee42d004825bbc322758eca0733 (127.24.0.129:41109), 2fb3a91da6ad4a2390d98ba13a53ed06 (127.24.0.131:35249)
I20260812 01:21:56.953896 18699 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c6f1fb89dde44aaa219c8fb0fb7f32c" candidate_uuid: "6cf4f241cc73458ea35718b385f9207f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fb3a91da6ad4a2390d98ba13a53ed06"
I20260812 01:21:56.953938 18227 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c6f1fb89dde44aaa219c8fb0fb7f32c" candidate_uuid: "6cf4f241cc73458ea35718b385f9207f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55139ee42d004825bbc322758eca0733"
I20260812 01:21:56.953979 18699 raft_consensus.cc:3037] T 1c6f1fb89dde44aaa219c8fb0fb7f32c P 2fb3a91da6ad4a2390d98ba13a53ed06 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:21:56.954023 18227 raft_consensus.cc:3037] T 1c6f1fb89dde44aaa219c8fb0fb7f32c P 55139ee42d004825bbc322758eca0733 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:21:56.954771 18699 raft_consensus.cc:2454] T 1c6f1fb89dde44aaa219c8fb0fb7f32c P 2fb3a91da6ad4a2390d98ba13a53ed06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6cf4f241cc73458ea35718b385f9207f in term 1.
I20260812 01:21:56.954798 18227 raft_consensus.cc:2454] T 1c6f1fb89dde44aaa219c8fb0fb7f32c P 55139ee42d004825bbc322758eca0733 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6cf4f241cc73458ea35718b385f9207f in term 1.
I20260812 01:21:56.954962 18336 leader_election.cc:302] T 1c6f1fb89dde44aaa219c8fb0fb7f32c P 6cf4f241cc73458ea35718b385f9207f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55139ee42d004825bbc322758eca0733, 6cf4f241cc73458ea35718b385f9207f; no voters:
I20260812 01:21:56.955096 18822 raft_consensus.cc:2781] T 1c6f1fb89dde44aaa219c8fb0fb7f32c P 6cf4f241cc73458ea35718b385f9207f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:21:56.955297 18822 raft_consensus.cc:686] T 1c6f1fb89dde44aaa219c8fb0fb7f32c P 6cf4f241cc73458ea35718b385f9207f [term 1 LEADER]: Becoming Leader. State: Replica: 6cf4f241cc73458ea35718b385f9207f, State: Running, Role: LEADER
I20260812 01:21:56.955401 18822 consensus_queue.cc:237] T 1c6f1fb89dde44aaa219c8fb0fb7f32c P 6cf4f241cc73458ea35718b385f9207f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55139ee42d004825bbc322758eca0733" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 41109 } } peers { permanent_uuid: "2fb3a91da6ad4a2390d98ba13a53ed06" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 35249 } } peers { permanent_uuid: "6cf4f241cc73458ea35718b385f9207f" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39345 } }
I20260812 01:21:56.956277 17968 catalog_manager.cc:5495] T 1c6f1fb89dde44aaa219c8fb0fb7f32c P 6cf4f241cc73458ea35718b385f9207f reported cstate change: term changed from 0 to 1, leader changed from <none> to 6cf4f241cc73458ea35718b385f9207f (127.24.0.130). New cstate: current_term: 1 leader_uuid: "6cf4f241cc73458ea35718b385f9207f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55139ee42d004825bbc322758eca0733" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 41109 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2fb3a91da6ad4a2390d98ba13a53ed06" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 35249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6cf4f241cc73458ea35718b385f9207f" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39345 } health_report { overall_health: HEALTHY } } }
W20260812 01:21:56.990341 18771 tablet.cc:2369] T 1c6f1fb89dde44aaa219c8fb0fb7f32c P 2fb3a91da6ad4a2390d98ba13a53ed06: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 01:21:56.998322 18798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:56.995967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57054 (local address 127.24.0.130:39345)
0812 01:21:56.996116 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0812 01:21:56.996118 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:21:56.996135 (+ 17us) server_negotiation.cc:407] Connection header received
0812 01:21:56.996175 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:21:56.996176 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:21:56.996198 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:21:56.996267 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:21:56.996397 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:56.996865 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:56.997585 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:21:56.997780 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:21:56.997846 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:21:56.997973 (+ 127us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:21:56.998073 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:21:56.998156 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 01:21:56.998189 (+ 33us) server_negotiation.cc:299] Negotiation successful
0812 01:21:56.998208 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260812 01:21:57.000229 18699 raft_consensus.cc:1261] T 1c6f1fb89dde44aaa219c8fb0fb7f32c P 2fb3a91da6ad4a2390d98ba13a53ed06 [term 1 FOLLOWER]: Refusing update from remote peer 6cf4f241cc73458ea35718b385f9207f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:21:57.000243 18227 raft_consensus.cc:1261] T 1c6f1fb89dde44aaa219c8fb0fb7f32c P 55139ee42d004825bbc322758eca0733 [term 1 FOLLOWER]: Refusing update from remote peer 6cf4f241cc73458ea35718b385f9207f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:21:57.000576 18825 consensus_queue.cc:1035] T 1c6f1fb89dde44aaa219c8fb0fb7f32c P 6cf4f241cc73458ea35718b385f9207f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2fb3a91da6ad4a2390d98ba13a53ed06" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 35249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:21:57.000720 18822 consensus_queue.cc:1035] T 1c6f1fb89dde44aaa219c8fb0fb7f32c P 6cf4f241cc73458ea35718b385f9207f [LEADER]: Connected to new peer: Peer: permanent_uuid: "55139ee42d004825bbc322758eca0733" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 41109 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:21:57.003140 18830 mvcc.cc:205] Tried to move back new op lower bound from 7317494648829906944 to 7317494648649854976. Current Snapshot: MvccSnapshot[applied={T|T < 7317494648829906944}]
I20260812 01:21:57.003845 18832 mvcc.cc:205] Tried to move back new op lower bound from 7317494648829906944 to 7317494648649854976. Current Snapshot: MvccSnapshot[applied={T|T < 7317494648829906944}]
I20260812 01:21:57.004014 18831 mvcc.cc:205] Tried to move back new op lower bound from 7317494648829906944 to 7317494648649854976. Current Snapshot: MvccSnapshot[applied={T|T < 7317494648829906944 or (T in {7317494648829906944})}]
I20260812 01:21:57.005591 17968 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40270:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:21:57.005698 17968 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40270:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:21:57.006451 17968 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 30cc471efc684639a2d06046514aaf95: Sending DeleteTablet for 3 replicas of tablet 1c6f1fb89dde44aaa219c8fb0fb7f32c
I20260812 01:21:57.006773 18194 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c6f1fb89dde44aaa219c8fb0fb7f32c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:21:57 UTC) from {username='kudu', principal='kudu/127.24.0.190@KRBTEST.COM'} at 127.0.0.1:58182
I20260812 01:21:57.006788 18434 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c6f1fb89dde44aaa219c8fb0fb7f32c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:21:57 UTC) from {username='kudu', principal='kudu/127.24.0.190@KRBTEST.COM'} at 127.0.0.1:57052
I20260812 01:21:57.007025 18665 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c6f1fb89dde44aaa219c8fb0fb7f32c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:21:57 UTC) from {username='kudu', principal='kudu/127.24.0.190@KRBTEST.COM'} at 127.0.0.1:44324
I20260812 01:21:57.007253 18850 tablet_replica.cc:330] T 1c6f1fb89dde44aaa219c8fb0fb7f32c P 2fb3a91da6ad4a2390d98ba13a53ed06: stopping tablet replica
I20260812 01:21:57.007325 18847 tablet_replica.cc:330] T 1c6f1fb89dde44aaa219c8fb0fb7f32c P 6cf4f241cc73458ea35718b385f9207f: stopping tablet replica
I20260812 01:21:57.007344 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 18080
I20260812 01:21:57.007413 18850 raft_consensus.cc:2229] T 1c6f1fb89dde44aaa219c8fb0fb7f32c P 2fb3a91da6ad4a2390d98ba13a53ed06 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 01:21:57.015436 18850 raft_consensus.cc:2258] T 1c6f1fb89dde44aaa219c8fb0fb7f32c P 2fb3a91da6ad4a2390d98ba13a53ed06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:21:57.024994 18847 raft_consensus.cc:2229] T 1c6f1fb89dde44aaa219c8fb0fb7f32c P 6cf4f241cc73458ea35718b385f9207f [term 1 LEADER]: Raft consensus shutting down.
I20260812 01:21:57.025139 18847 raft_consensus.cc:2258] T 1c6f1fb89dde44aaa219c8fb0fb7f32c P 6cf4f241cc73458ea35718b385f9207f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:21:57.026126 18850 ts_tablet_manager.cc:1905] T 1c6f1fb89dde44aaa219c8fb0fb7f32c P 2fb3a91da6ad4a2390d98ba13a53ed06: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:21:57.026126 18847 ts_tablet_manager.cc:1905] T 1c6f1fb89dde44aaa219c8fb0fb7f32c P 6cf4f241cc73458ea35718b385f9207f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:21:57.028155 18850 ts_tablet_manager.cc:1918] T 1c6f1fb89dde44aaa219c8fb0fb7f32c P 2fb3a91da6ad4a2390d98ba13a53ed06: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 01:21:57.028185 18847 ts_tablet_manager.cc:1918] T 1c6f1fb89dde44aaa219c8fb0fb7f32c P 6cf4f241cc73458ea35718b385f9207f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 01:21:57.028262 18847 log.cc:1196] T 1c6f1fb89dde44aaa219c8fb0fb7f32c P 6cf4f241cc73458ea35718b385f9207f: Deleting WAL directory at /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-1/wal/wals/1c6f1fb89dde44aaa219c8fb0fb7f32c
I20260812 01:21:57.028331 18850 log.cc:1196] T 1c6f1fb89dde44aaa219c8fb0fb7f32c P 2fb3a91da6ad4a2390d98ba13a53ed06: Deleting WAL directory at /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-2/wal/wals/1c6f1fb89dde44aaa219c8fb0fb7f32c
I20260812 01:21:57.028664 18850 ts_tablet_manager.cc:1939] T 1c6f1fb89dde44aaa219c8fb0fb7f32c P 2fb3a91da6ad4a2390d98ba13a53ed06: Deleting consensus metadata
I20260812 01:21:57.028729 18847 ts_tablet_manager.cc:1939] T 1c6f1fb89dde44aaa219c8fb0fb7f32c P 6cf4f241cc73458ea35718b385f9207f: Deleting consensus metadata
I20260812 01:21:57.029217 17948 catalog_manager.cc:4841] TS 2fb3a91da6ad4a2390d98ba13a53ed06 (127.24.0.131:35249): tablet 1c6f1fb89dde44aaa219c8fb0fb7f32c (table test-table [id=daab6165e8f54d67a06e4b1633f4ed53]) successfully deleted
I20260812 01:21:57.029230 17945 catalog_manager.cc:4841] TS 6cf4f241cc73458ea35718b385f9207f (127.24.0.130:39345): tablet 1c6f1fb89dde44aaa219c8fb0fb7f32c (table test-table [id=daab6165e8f54d67a06e4b1633f4ed53]) successfully deleted
W20260812 01:21:57.032325 17944 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.0.129:41109 (error 108)
I20260812 01:21:57.032939 18793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:21:57.032787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.129:41109 (local address 127.0.0.1:58190)
0812 01:21:57.032857 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0812 01:21:57.032901 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.129:41109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260812 01:21:57.033011 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 18311
W20260812 01:21:57.039521 17944 catalog_manager.cc:4584] TS 55139ee42d004825bbc322758eca0733 (127.24.0.129:41109): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1c6f1fb89dde44aaa219c8fb0fb7f32c: Network error: Client connection negotiation failed: client connection to 127.24.0.129:41109: connect: Connection refused (error 111)
I20260812 01:21:57.043892 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 18553
I20260812 01:21:57.048856 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 17916
I20260812 01:22:17.059825 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.190:37371
--webserver_interface=127.24.0.190
--webserver_port=43471
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab
--principal=kudu/127.24.0.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:40363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.0.190:37371
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:22:17.152372 18854 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:22:17.152622 18854 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:22:17.152658 18854 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:22:17.153931 18854 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260812 01:22:17.153972 18854 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260812 01:22:17.153986 18854 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260812 01:22:17.153998 18854 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:22:17.154049 18854 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:22:17.154068 18854 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:22:17.154094 18854 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:22:17.154124 18854 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:22:17.155951 18854 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:40363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.24.0.190:37371
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab
--principal=kudu/127.24.0.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.190:37371
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.0.190
--webserver_port=43471
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18854
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:22:17.156250 18854 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:22:17.156523 18854 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:22:17.166913 18868 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:22:17.166924 18867 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:22:17.167196 18875 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:22:17.167232 18854 server_base.cc:1034] running on GCE node
I20260812 01:22:17.167585 18854 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:22:17.167959 18854 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:22:17.169123 18854 hybrid_clock.cc:648] HybridClock initialized: now 1786497737169117 us; error 39 us; skew 500 ppm
Aug 12 01:22:17 dist-test-slave-t3q3 krb5kdc[17894](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497737, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:22:17.172466 18854 init.cc:377] Logged in from keytab as kudu/127.24.0.190@KRBTEST.COM (short username kudu)
I20260812 01:22:17.173064 18854 webserver.cc:460] Webserver started at http://127.24.0.190:43471/ using document root <none> and password file <none>
I20260812 01:22:17.173363 18854 fs_manager.cc:359] Metadata directory not provided
I20260812 01:22:17.173437 18854 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:22:17.175206 18854 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 01:22:17.176025 18890 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:22:17.176290 18854 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:22:17.176389 18854 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/master-0/wal
uuid: "30cc471efc684639a2d06046514aaf95"
format_stamp: "Formatted at 2026-08-12 01:21:56 on dist-test-slave-t3q3"
server_key: "6c3c8d0e32baad54521b8a63f0346eae"
server_key_iv: "2784481ffa70883c7439aee8d9d1c5c8"
server_key_version: "encryptionkey@0"
I20260812 01:22:17.176903 18854 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:22:17.190716 18854 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:22:17.192903 18854 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:22:17.193084 18854 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:22:17.197785 18854 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.190:37371
I20260812 01:22:17.197842 18974 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.190:37371 every 8 connection(s)
I20260812 01:22:17.198261 18854 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
I20260812 01:22:17.199846 18975 sys_catalog.cc:263] Verifying existing consensus state
I20260812 01:22:17.200621 18975 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30cc471efc684639a2d06046514aaf95: Bootstrap starting.
I20260812 01:22:17.207616 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 18854
I20260812 01:22:17.207993 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.0.129:41109
--local_ip_for_outbound_sockets=127.24.0.129
--tserver_master_addrs=127.24.0.190:37371
--webserver_port=36545
--webserver_interface=127.24.0.129
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.129.keytab
--principal=kudu/127.24.0.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:40363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 01:22:17.210513 18975 log.cc:824] T 00000000000000000000000000000000 P 30cc471efc684639a2d06046514aaf95: Log is configured to *not* fsync() on all Append() calls
I20260812 01:22:17.212647 18975 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30cc471efc684639a2d06046514aaf95: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 01:22:17.213022 18975 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30cc471efc684639a2d06046514aaf95: Bootstrap complete.
I20260812 01:22:17.215402 18975 raft_consensus.cc:348] T 00000000000000000000000000000000 P 30cc471efc684639a2d06046514aaf95 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30cc471efc684639a2d06046514aaf95" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 37371 } }
I20260812 01:22:17.215633 18975 raft_consensus.cc:729] T 00000000000000000000000000000000 P 30cc471efc684639a2d06046514aaf95 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30cc471efc684639a2d06046514aaf95, State: Initialized, Role: FOLLOWER
I20260812 01:22:17.215816 18975 consensus_queue.cc:260] T 00000000000000000000000000000000 P 30cc471efc684639a2d06046514aaf95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30cc471efc684639a2d06046514aaf95" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 37371 } }
I20260812 01:22:17.215883 18975 raft_consensus.cc:388] T 00000000000000000000000000000000 P 30cc471efc684639a2d06046514aaf95 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:22:17.215951 18975 raft_consensus.cc:482] T 00000000000000000000000000000000 P 30cc471efc684639a2d06046514aaf95 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:22:17.216005 18975 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 30cc471efc684639a2d06046514aaf95 [term 1 FOLLOWER]: Advancing to term 2
I20260812 01:22:17.217105 18975 raft_consensus.cc:504] T 00000000000000000000000000000000 P 30cc471efc684639a2d06046514aaf95 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30cc471efc684639a2d06046514aaf95" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 37371 } }
I20260812 01:22:17.217201 18975 leader_election.cc:302] T 00000000000000000000000000000000 P 30cc471efc684639a2d06046514aaf95 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 30cc471efc684639a2d06046514aaf95; no voters:
I20260812 01:22:17.217442 18975 leader_election.cc:288] T 00000000000000000000000000000000 P 30cc471efc684639a2d06046514aaf95 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 01:22:17.217480 18981 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 30cc471efc684639a2d06046514aaf95 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 01:22:17.217608 18981 raft_consensus.cc:686] T 00000000000000000000000000000000 P 30cc471efc684639a2d06046514aaf95 [term 2 LEADER]: Becoming Leader. State: Replica: 30cc471efc684639a2d06046514aaf95, State: Running, Role: LEADER
I20260812 01:22:17.217743 18975 sys_catalog.cc:564] T 00000000000000000000000000000000 P 30cc471efc684639a2d06046514aaf95 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:22:17.217767 18981 consensus_queue.cc:237] T 00000000000000000000000000000000 P 30cc471efc684639a2d06046514aaf95 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30cc471efc684639a2d06046514aaf95" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 37371 } }
I20260812 01:22:17.218053 18981 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30cc471efc684639a2d06046514aaf95 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 30cc471efc684639a2d06046514aaf95. Latest consensus state: current_term: 2 leader_uuid: "30cc471efc684639a2d06046514aaf95" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30cc471efc684639a2d06046514aaf95" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 37371 } } }
I20260812 01:22:17.218310 18981 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30cc471efc684639a2d06046514aaf95 [sys.catalog]: This master's current role is: LEADER
I20260812 01:22:17.218220 18982 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30cc471efc684639a2d06046514aaf95 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "30cc471efc684639a2d06046514aaf95" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30cc471efc684639a2d06046514aaf95" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 37371 } } }
I20260812 01:22:17.218758 18982 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30cc471efc684639a2d06046514aaf95 [sys.catalog]: This master's current role is: LEADER
I20260812 01:22:17.219043 19002 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:22:17.219815 19002 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:22:17.219895 19002 catalog_manager.cc:1259] Loaded cluster ID: d4417e3d8d03430ab68a6c014449d70a
I20260812 01:22:17.219954 19002 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 01:22:17.221185 19002 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:22:17.221385 19002 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 30cc471efc684639a2d06046514aaf95: Loaded TSK: 0
I20260812 01:22:17.237442 19002 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 30cc471efc684639a2d06046514aaf95: Generated new TSK 1
I20260812 01:22:17.237545 19002 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 01:22:17.296756 18978 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:22:17.297008 18978 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:22:17.297048 18978 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:22:17.298239 18978 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:22:17.298311 18978 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:22:17.298357 18978 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.0.129
I20260812 01:22:17.299868 18978 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:40363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.129.keytab
--principal=kudu/127.24.0.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.0.129:41109
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.0.129
--webserver_port=36545
--tserver_master_addrs=127.24.0.190:37371
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18978
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.0.129
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:22:17.300182 18978 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:22:17.300441 18978 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:22:17.306569 19027 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:22:17.306581 19029 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:22:17.306622 19033 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:22:17.306914 18978 server_base.cc:1034] running on GCE node
I20260812 01:22:17.307117 18978 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:22:17.307384 18978 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:22:17.308561 18978 hybrid_clock.cc:648] HybridClock initialized: now 1786497737308547 us; error 31 us; skew 500 ppm
Aug 12 01:22:17 dist-test-slave-t3q3 krb5kdc[17894](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497737, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:22:17.311338 18978 init.cc:377] Logged in from keytab as kudu/127.24.0.129@KRBTEST.COM (short username kudu)
I20260812 01:22:17.311892 18978 webserver.cc:460] Webserver started at http://127.24.0.129:36545/ using document root <none> and password file <none>
I20260812 01:22:17.312203 18978 fs_manager.cc:359] Metadata directory not provided
I20260812 01:22:17.312269 18978 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:22:17.313747 18978 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 01:22:17.314523 19042 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:22:17.314718 18978 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 01:22:17.314811 18978 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-0/wal
uuid: "55139ee42d004825bbc322758eca0733"
format_stamp: "Formatted at 2026-08-12 01:21:56 on dist-test-slave-t3q3"
server_key: "e505175fce3d46c439ffb1acc925b3aa"
server_key_iv: "5824cdee799dda6087b788590ad2a7ad"
server_key_version: "encryptionkey@0"
I20260812 01:22:17.315176 18978 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:22:17.333593 18978 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:22:17.336236 18978 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:22:17.336400 18978 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:22:17.336701 18978 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:22:17.337297 19056 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 01:22:17.338853 18978 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 01:22:17.338910 18978 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260812 01:22:17.338946 18978 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 01:22:17.339692 18978 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 01:22:17.339741 18978 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260812 01:22:17.339792 19056 tablet_bootstrap.cc:492] T 1c6f1fb89dde44aaa219c8fb0fb7f32c P 55139ee42d004825bbc322758eca0733: Bootstrap starting.
I20260812 01:22:17.348260 18978 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.129:41109
I20260812 01:22:17.348351 19259 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.129:41109 every 8 connection(s)
I20260812 01:22:17.348979 18978 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-0/data/info.pb
I20260812 01:22:17.352640 19056 log.cc:824] T 1c6f1fb89dde44aaa219c8fb0fb7f32c P 55139ee42d004825bbc322758eca0733: Log is configured to *not* fsync() on all Append() calls
I20260812 01:22:17.355154 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 18978
I20260812 01:22:17.355466 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.0.130:39345
--local_ip_for_outbound_sockets=127.24.0.130
--tserver_master_addrs=127.24.0.190:37371
--webserver_port=41971
--webserver_interface=127.24.0.130
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.130.keytab
--principal=kudu/127.24.0.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:40363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 01:22:17.355598 19056 tablet_bootstrap.cc:492] T 1c6f1fb89dde44aaa219c8fb0fb7f32c P 55139ee42d004825bbc322758eca0733: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 01:22:17.356415 19056 tablet_bootstrap.cc:492] T 1c6f1fb89dde44aaa219c8fb0fb7f32c P 55139ee42d004825bbc322758eca0733: Bootstrap complete.
I20260812 01:22:17.357709 19056 ts_tablet_manager.cc:1397] T 1c6f1fb89dde44aaa219c8fb0fb7f32c P 55139ee42d004825bbc322758eca0733: Time spent bootstrapping tablet: real 0.018s user 0.012s sys 0.004s
I20260812 01:22:17.359114 19056 raft_consensus.cc:348] T 1c6f1fb89dde44aaa219c8fb0fb7f32c P 55139ee42d004825bbc322758eca0733 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55139ee42d004825bbc322758eca0733" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 41109 } } peers { permanent_uuid: "2fb3a91da6ad4a2390d98ba13a53ed06" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 35249 } } peers { permanent_uuid: "6cf4f241cc73458ea35718b385f9207f" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39345 } }
I20260812 01:22:17.359468 19056 raft_consensus.cc:729] T 1c6f1fb89dde44aaa219c8fb0fb7f32c P 55139ee42d004825bbc322758eca0733 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55139ee42d004825bbc322758eca0733, State: Initialized, Role: FOLLOWER
Aug 12 01:22:17 dist-test-slave-t3q3 krb5kdc[17894](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497737, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.129@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
I20260812 01:22:17.359606 19056 consensus_queue.cc:260] T 1c6f1fb89dde44aaa219c8fb0fb7f32c P 55139ee42d004825bbc322758eca0733 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55139ee42d004825bbc322758eca0733" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 41109 } } peers { permanent_uuid: "2fb3a91da6ad4a2390d98ba13a53ed06" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 35249 } } peers { permanent_uuid: "6cf4f241cc73458ea35718b385f9207f" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39345 } }
I20260812 01:22:17.359928 19056 ts_tablet_manager.cc:1428] T 1c6f1fb89dde44aaa219c8fb0fb7f32c P 55139ee42d004825bbc322758eca0733: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260812 01:22:17.364111 19266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:17.349763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:37371 (local address 127.24.0.129:40163)
0812 01:22:17.349998 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0812 01:22:17.350047 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 01:22:17.350668 (+ 621us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:22:17.352772 (+ 2104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:22:17.352777 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:22:17.353203 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:22:17.353474 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:22:17.353482 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:22:17.355151 (+ 1669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:22:17.355152 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:22:17.356468 (+ 1316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:22:17.356472 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:22:17.357359 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:22:17.357361 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:22:17.357532 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:22:17.358148 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:22:17.358160 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:22:17.360196 (+ 2036us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:22:17.362283 (+ 2087us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:22:17.362292 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:22:17.362294 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:22:17.362533 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:22:17.362830 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:22:17.362831 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:22:17.362832 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:22:17.362981 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:22:17.363546 (+ 565us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:22:17.363551 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:22:17.363684 (+ 133us) client_negotiation.cc:769] Sending connection context
0812 01:22:17.363772 (+ 88us) client_negotiation.cc:240] Negotiation successful
0812 01:22:17.363910 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":135,"threads_started":1}
I20260812 01:22:17.364321 19267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:17.349913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.129:40163 (local address 127.24.0.190:37371)
0812 01:22:17.352246 (+ 2333us) server_negotiation.cc:206] Beginning negotiation
0812 01:22:17.352257 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 01:22:17.352307 (+ 50us) server_negotiation.cc:407] Connection header received
0812 01:22:17.352468 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:22:17.352478 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:22:17.352544 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:22:17.352711 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:22:17.353626 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:17.354985 (+ 1359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:17.356597 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:17.357243 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:17.357370 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:22:17.360315 (+ 2945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:22:17.360320 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:22:17.360321 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:22:17.360334 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:22:17.362188 (+ 1854us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:22:17.362646 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:22:17.362650 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:22:17.362650 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:22:17.362752 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:22:17.363084 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:22:17.363086 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:22:17.363086 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:22:17.363456 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:22:17.363568 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0812 01:22:17.363881 (+ 313us) server_negotiation.cc:299] Negotiation successful
0812 01:22:17.364081 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2261,"thread_start_us":84,"threads_started":1}
I20260812 01:22:17.365144 19262 heartbeater.cc:347] Connected to a master server at 127.24.0.190:37371
I20260812 01:22:17.365262 19262 heartbeater.cc:464] Registering TS with master...
I20260812 01:22:17.365465 19262 heartbeater.cc:511] Master 127.24.0.190:37371 requested a full tablet report, sending...
I20260812 01:22:17.366235 18918 ts_manager.cc:194] Registered new tserver with Master: 55139ee42d004825bbc322758eca0733 (127.24.0.129:41109)
I20260812 01:22:17.367367 18918 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.0.129@KRBTEST.COM'} at 127.24.0.129:40163
I20260812 01:22:17.368722 19262 heartbeater.cc:503] Master 127.24.0.190:37371 was elected leader, sending a full tablet report...
I20260812 01:22:17.372128 19277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:17.366866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.129:41109 (local address 127.0.0.1:35204)
0812 01:22:17.368183 (+ 1317us) negotiation.cc:107] Waiting for socket to connect
0812 01:22:17.368230 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 01:22:17.368318 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:22:17.368702 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:22:17.368706 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:22:17.368709 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:22:17.368783 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:22:17.368786 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:22:17.369809 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:22:17.369811 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:22:17.371134 (+ 1323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:22:17.371137 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:22:17.371792 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:22:17.371794 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:22:17.371883 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:22:17.371898 (+ 15us) client_negotiation.cc:769] Sending connection context
0812 01:22:17.371973 (+ 75us) client_negotiation.cc:240] Negotiation successful
0812 01:22:17.372033 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1278,"thread_start_us":91,"threads_started":1}
I20260812 01:22:17.372141 19280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:17.366953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35204 (local address 127.24.0.129:41109)
0812 01:22:17.367121 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0812 01:22:17.367126 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 01:22:17.368409 (+ 1283us) server_negotiation.cc:407] Connection header received
0812 01:22:17.368453 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:22:17.368456 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:22:17.368573 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:22:17.368679 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:22:17.368888 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:17.369725 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:17.371227 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:17.371708 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:17.371773 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:22:17.371858 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 01:22:17.371995 (+ 137us) server_negotiation.cc:299] Negotiation successful
0812 01:22:17.372028 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":100,"threads_started":1}
I20260812 01:22:17.372548 19142 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c6f1fb89dde44aaa219c8fb0fb7f32c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:21:57 UTC) from {username='kudu', principal='kudu/127.24.0.190@KRBTEST.COM'} at 127.0.0.1:35204
I20260812 01:22:17.373198 19281 tablet_replica.cc:330] stopping tablet replica
I20260812 01:22:17.373327 19281 raft_consensus.cc:2229] T 1c6f1fb89dde44aaa219c8fb0fb7f32c P 55139ee42d004825bbc322758eca0733 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 01:22:17.373400 19281 raft_consensus.cc:2258] T 1c6f1fb89dde44aaa219c8fb0fb7f32c P 55139ee42d004825bbc322758eca0733 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:22:17.373700 19281 ts_tablet_manager.cc:1905] T 1c6f1fb89dde44aaa219c8fb0fb7f32c P 55139ee42d004825bbc322758eca0733: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:22:17.375762 19281 ts_tablet_manager.cc:1918] T 1c6f1fb89dde44aaa219c8fb0fb7f32c P 55139ee42d004825bbc322758eca0733: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 01:22:17.375883 19281 log.cc:1196] T 1c6f1fb89dde44aaa219c8fb0fb7f32c P 55139ee42d004825bbc322758eca0733: Deleting WAL directory at /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-0/wal/wals/1c6f1fb89dde44aaa219c8fb0fb7f32c
I20260812 01:22:17.376222 19281 ts_tablet_manager.cc:1939] T 1c6f1fb89dde44aaa219c8fb0fb7f32c P 55139ee42d004825bbc322758eca0733: Deleting consensus metadata
I20260812 01:22:17.376798 18891 catalog_manager.cc:4841] TS 55139ee42d004825bbc322758eca0733 (127.24.0.129:41109): tablet 1c6f1fb89dde44aaa219c8fb0fb7f32c (table test-table [id=daab6165e8f54d67a06e4b1633f4ed53]) successfully deleted
W20260812 01:22:17.445869 19273 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:22:17.446085 19273 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:22:17.446110 19273 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:22:17.447291 19273 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:22:17.447335 19273 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:22:17.447362 19273 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.0.130
I20260812 01:22:17.448853 19273 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:40363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.130.keytab
--principal=kudu/127.24.0.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.0.130:39345
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.0.130
--webserver_port=41971
--tserver_master_addrs=127.24.0.190:37371
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19273
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.0.130
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:22:17.449120 19273 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:22:17.449368 19273 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:22:17.454576 19288 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:22:17.454810 19273 server_base.cc:1034] running on GCE node
W20260812 01:22:17.454559 19290 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:22:17.454552 19292 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:22:17.455124 19273 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:22:17.455412 19273 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:22:17.456581 19273 hybrid_clock.cc:648] HybridClock initialized: now 1786497737456575 us; error 35 us; skew 500 ppm
Aug 12 01:22:17 dist-test-slave-t3q3 krb5kdc[17894](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497737, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:22:17.459134 19273 init.cc:377] Logged in from keytab as kudu/127.24.0.130@KRBTEST.COM (short username kudu)
I20260812 01:22:17.459630 19273 webserver.cc:460] Webserver started at http://127.24.0.130:41971/ using document root <none> and password file <none>
I20260812 01:22:17.459887 19273 fs_manager.cc:359] Metadata directory not provided
I20260812 01:22:17.459976 19273 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:22:17.461447 19273 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 01:22:17.462144 19302 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:22:17.462378 19273 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 01:22:17.462487 19273 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-1/wal
uuid: "6cf4f241cc73458ea35718b385f9207f"
format_stamp: "Formatted at 2026-08-12 01:21:56 on dist-test-slave-t3q3"
server_key: "7961aa150737e2fcea0b71ba23418d1e"
server_key_iv: "6d79c2a710d80a3af872ef54a5d36a45"
server_key_version: "encryptionkey@0"
I20260812 01:22:17.462846 19273 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:22:17.470615 19273 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:22:17.473083 19273 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:22:17.473258 19273 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:22:17.473515 19273 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:22:17.473843 19273 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:22:17.473899 19273 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:22:17.473941 19273 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:22:17.473970 19273 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:22:17.480365 19273 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.130:39345
I20260812 01:22:17.480428 19490 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.130:39345 every 8 connection(s)
I20260812 01:22:17.480780 19273 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-1/data/info.pb
I20260812 01:22:17.483726 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 19273
I20260812 01:22:17.484016 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.0.131:35249
--local_ip_for_outbound_sockets=127.24.0.131
--tserver_master_addrs=127.24.0.190:37371
--webserver_port=33579
--webserver_interface=127.24.0.131
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.131.keytab
--principal=kudu/127.24.0.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:40363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 01:22:17 dist-test-slave-t3q3 krb5kdc[17894](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497737, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.130@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
I20260812 01:22:17.491261 19267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:17.481569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.130:33187 (local address 127.24.0.190:37371)
0812 01:22:17.481636 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0812 01:22:17.481639 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:22:17.481909 (+ 270us) server_negotiation.cc:407] Connection header received
0812 01:22:17.482418 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:22:17.482420 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:22:17.482435 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:22:17.482488 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:22:17.483439 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:17.484026 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:17.485610 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:17.485825 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:17.485947 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:22:17.488287 (+ 2340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:22:17.488289 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:22:17.488289 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:22:17.488297 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:22:17.489794 (+ 1497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:22:17.490241 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:22:17.490242 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:22:17.490242 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:22:17.490308 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:22:17.490572 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:22:17.490573 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:22:17.490573 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:22:17.490699 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:22:17.490773 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 01:22:17.491063 (+ 290us) server_negotiation.cc:299] Negotiation successful
0812 01:22:17.491159 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 01:22:17.491447 19497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:17.481556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:37371 (local address 127.24.0.130:33187)
0812 01:22:17.481777 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0812 01:22:17.481817 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 01:22:17.482326 (+ 509us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:22:17.482548 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:22:17.482552 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:22:17.483145 (+ 593us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:22:17.483360 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:22:17.483365 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:22:17.484199 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:22:17.484200 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:22:17.485492 (+ 1292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:22:17.485496 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:22:17.485959 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:22:17.485960 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:22:17.486100 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:22:17.486666 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:22:17.486675 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:22:17.488182 (+ 1507us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:22:17.489900 (+ 1718us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:22:17.489908 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:22:17.489910 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:22:17.490141 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:22:17.490377 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:22:17.490378 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:22:17.490379 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:22:17.490501 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:22:17.490777 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:22:17.490785 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:22:17.490929 (+ 144us) client_negotiation.cc:769] Sending connection context
0812 01:22:17.491040 (+ 111us) client_negotiation.cc:240] Negotiation successful
0812 01:22:17.491202 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":134,"threads_started":1}
I20260812 01:22:17.491936 19492 heartbeater.cc:347] Connected to a master server at 127.24.0.190:37371
I20260812 01:22:17.492089 19492 heartbeater.cc:464] Registering TS with master...
I20260812 01:22:17.492314 19492 heartbeater.cc:511] Master 127.24.0.190:37371 requested a full tablet report, sending...
I20260812 01:22:17.492718 18918 ts_manager.cc:194] Registered new tserver with Master: 6cf4f241cc73458ea35718b385f9207f (127.24.0.130:39345)
I20260812 01:22:17.493117 18918 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.0.130@KRBTEST.COM'} at 127.24.0.130:33187
W20260812 01:22:17.570461 19498 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:22:17.570689 19498 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:22:17.570729 19498 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:22:17.571873 19498 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:22:17.571938 19498 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:22:17.571985 19498 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.0.131
I20260812 01:22:17.573748 19498 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:40363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.131.keytab
--principal=kudu/127.24.0.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.0.131:35249
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.0.131
--webserver_port=33579
--tserver_master_addrs=127.24.0.190:37371
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19498
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.0.131
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:22:17.574015 19498 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:22:17.574272 19498 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:22:17.579978 19511 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:22:17.580008 19507 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:22:17.580049 19506 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:22:17.580294 19498 server_base.cc:1034] running on GCE node
I20260812 01:22:17.580530 19498 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:22:17.580791 19498 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:22:17.581933 19498 hybrid_clock.cc:648] HybridClock initialized: now 1786497737581921 us; error 28 us; skew 500 ppm
Aug 12 01:22:17 dist-test-slave-t3q3 krb5kdc[17894](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497737, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:22:17.584409 19498 init.cc:377] Logged in from keytab as kudu/127.24.0.131@KRBTEST.COM (short username kudu)
I20260812 01:22:17.584800 19498 webserver.cc:460] Webserver started at http://127.24.0.131:33579/ using document root <none> and password file <none>
I20260812 01:22:17.585006 19498 fs_manager.cc:359] Metadata directory not provided
I20260812 01:22:17.585060 19498 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:22:17.586346 19498 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 01:22:17.586905 19528 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:22:17.587073 19498 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 01:22:17.587165 19498 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-2/wal
uuid: "2fb3a91da6ad4a2390d98ba13a53ed06"
format_stamp: "Formatted at 2026-08-12 01:21:56 on dist-test-slave-t3q3"
server_key: "2984abd5ee12c68ca949e6f9be6adc19"
server_key_iv: "4acc62bdf1abc43efab71c1f259d601a"
server_key_version: "encryptionkey@0"
I20260812 01:22:17.587527 19498 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:22:17.599210 19498 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:22:17.601454 19498 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:22:17.601629 19498 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:22:17.601887 19498 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:22:17.602219 19498 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:22:17.602275 19498 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:22:17.602319 19498 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:22:17.602348 19498 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:22:17.608583 19498 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.131:35249
I20260812 01:22:17.608644 19705 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.131:35249 every 8 connection(s)
I20260812 01:22:17.609010 19498 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-2/data/info.pb
I20260812 01:22:17.610782 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 19498
Aug 12 01:22:17 dist-test-slave-t3q3 krb5kdc[17894](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497737, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.131@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
I20260812 01:22:17.619475 19711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:17.611250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57622 (local address 127.24.0.190:37371)
0812 01:22:17.611369 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0812 01:22:17.611372 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:22:17.611510 (+ 138us) server_negotiation.cc:407] Connection header received
0812 01:22:17.611598 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:22:17.611599 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:22:17.611611 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:22:17.611690 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:22:17.612490 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:17.612971 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:17.613785 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:17.613989 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:17.614084 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:22:17.615469 (+ 1385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:22:17.615471 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:22:17.615472 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:22:17.615479 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:22:17.617726 (+ 2247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:22:17.618500 (+ 774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:22:17.618501 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:22:17.618501 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:22:17.618573 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:22:17.618873 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:22:17.618874 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:22:17.618875 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:22:17.618999 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:22:17.619229 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0812 01:22:17.619269 (+ 40us) server_negotiation.cc:299] Negotiation successful
0812 01:22:17.619353 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":72,"threads_started":1}
I20260812 01:22:17.619601 19267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:17.609796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.131:45411 (local address 127.24.0.190:37371)
0812 01:22:17.609862 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0812 01:22:17.609865 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:22:17.610127 (+ 262us) server_negotiation.cc:407] Connection header received
0812 01:22:17.610697 (+ 570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:22:17.610698 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:22:17.610713 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:22:17.610773 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:22:17.612023 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:17.612562 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:17.613609 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:17.613828 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:17.613906 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:22:17.616334 (+ 2428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:22:17.616335 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:22:17.616336 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:22:17.616343 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:22:17.618055 (+ 1712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:22:17.618483 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:22:17.618484 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:22:17.618484 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:22:17.618547 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:22:17.618931 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:22:17.618932 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:22:17.618933 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:22:17.619048 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:22:17.619125 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 01:22:17.619430 (+ 305us) server_negotiation.cc:299] Negotiation successful
0812 01:22:17.619516 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 01:22:17.619809 19710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:17.609764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:37371 (local address 127.24.0.131:45411)
0812 01:22:17.609961 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0812 01:22:17.610024 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0812 01:22:17.610571 (+ 547us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:22:17.610845 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:22:17.610851 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:22:17.611639 (+ 788us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:22:17.611914 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:22:17.611923 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:22:17.612664 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:22:17.612665 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:22:17.613512 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:22:17.613514 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:22:17.613911 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:22:17.613912 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:22:17.614053 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:22:17.614625 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:22:17.614638 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:22:17.616250 (+ 1612us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:22:17.618119 (+ 1869us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:22:17.618126 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:22:17.618129 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:22:17.618429 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:22:17.618648 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:22:17.618649 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:22:17.618649 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:22:17.618840 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:22:17.619119 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:22:17.619126 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:22:17.619281 (+ 155us) client_negotiation.cc:769] Sending connection context
0812 01:22:17.619393 (+ 112us) client_negotiation.cc:240] Negotiation successful
0812 01:22:17.619558 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":98,"threads_started":1}
I20260812 01:22:17.620327 19706 heartbeater.cc:347] Connected to a master server at 127.24.0.190:37371
I20260812 01:22:17.620435 19706 heartbeater.cc:464] Registering TS with master...
I20260812 01:22:17.620637 19706 heartbeater.cc:511] Master 127.24.0.190:37371 requested a full tablet report, sending...
I20260812 01:22:17.621043 18918 ts_manager.cc:194] Registered new tserver with Master: 2fb3a91da6ad4a2390d98ba13a53ed06 (127.24.0.131:35249)
I20260812 01:22:17.621430 18918 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.0.131@KRBTEST.COM'} at 127.24.0.131:45411
I20260812 01:22:17.631449 24578 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 01:22:17.634223 19267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:17.631854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57628 (local address 127.24.0.190:37371)
0812 01:22:17.631914 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0812 01:22:17.631916 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:22:17.632112 (+ 196us) server_negotiation.cc:407] Connection header received
0812 01:22:17.632187 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:22:17.632188 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:22:17.632228 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:22:17.632298 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:22:17.632396 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:17.632872 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:17.633621 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:17.633799 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:17.633851 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:22:17.633987 (+ 136us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:22:17.634037 (+ 50us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0812 01:22:17.634170 (+ 133us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:57628: token expired
Metrics: {"server-negotiator.queue_time_us":33}
W20260812 01:22:17.634305 19267 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:57628: token expired
W20260812 01:22:17.634500 18777 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.24.0.190:37371)
I20260812 01:22:17.640793 19267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:17.634811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57642 (local address 127.24.0.190:37371)
0812 01:22:17.634877 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0812 01:22:17.634879 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:22:17.634899 (+ 20us) server_negotiation.cc:407] Connection header received
0812 01:22:17.634916 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:22:17.634917 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:22:17.634924 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:22:17.634963 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:22:17.635445 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:17.635876 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:17.636616 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:17.636785 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:17.636839 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:22:17.638032 (+ 1193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:22:17.638033 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:22:17.638034 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:22:17.638039 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:22:17.639499 (+ 1460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:22:17.639866 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:22:17.639867 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:22:17.639868 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:22:17.639932 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:22:17.640200 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:22:17.640201 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:22:17.640201 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:22:17.640358 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:22:17.640457 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 01:22:17.640567 (+ 110us) server_negotiation.cc:299] Negotiation successful
0812 01:22:17.640640 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 01:22:17.642282 18918 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57642:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 01:22:17.644124 18918 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 01:22:17.646878 19142 tablet_service.cc:1467] Processing CreateTablet for tablet a593d4f08adc482aac259684573a4907 (DEFAULT_TABLE table=test-table [id=19c40fb0bebb47a396928d8b2e2a00d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:22:17.647107 19142 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a593d4f08adc482aac259684573a4907. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:22:17.648653 19056 tablet_bootstrap.cc:492] T a593d4f08adc482aac259684573a4907 P 55139ee42d004825bbc322758eca0733: Bootstrap starting.
I20260812 01:22:17.649055 19056 tablet_bootstrap.cc:654] T a593d4f08adc482aac259684573a4907 P 55139ee42d004825bbc322758eca0733: Neither blocks nor log segments found. Creating new log.
I20260812 01:22:17.649710 19056 tablet_bootstrap.cc:492] T a593d4f08adc482aac259684573a4907 P 55139ee42d004825bbc322758eca0733: No bootstrap required, opened a new log
I20260812 01:22:17.649755 19056 ts_tablet_manager.cc:1397] T a593d4f08adc482aac259684573a4907 P 55139ee42d004825bbc322758eca0733: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 01:22:17.649865 19056 raft_consensus.cc:348] T a593d4f08adc482aac259684573a4907 P 55139ee42d004825bbc322758eca0733 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cf4f241cc73458ea35718b385f9207f" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39345 } } peers { permanent_uuid: "2fb3a91da6ad4a2390d98ba13a53ed06" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 35249 } } peers { permanent_uuid: "55139ee42d004825bbc322758eca0733" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 41109 } }
I20260812 01:22:17.649937 19056 raft_consensus.cc:374] T a593d4f08adc482aac259684573a4907 P 55139ee42d004825bbc322758eca0733 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:22:17.649986 19056 raft_consensus.cc:729] T a593d4f08adc482aac259684573a4907 P 55139ee42d004825bbc322758eca0733 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55139ee42d004825bbc322758eca0733, State: Initialized, Role: FOLLOWER
I20260812 01:22:17.650028 19056 consensus_queue.cc:260] T a593d4f08adc482aac259684573a4907 P 55139ee42d004825bbc322758eca0733 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cf4f241cc73458ea35718b385f9207f" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39345 } } peers { permanent_uuid: "2fb3a91da6ad4a2390d98ba13a53ed06" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 35249 } } peers { permanent_uuid: "55139ee42d004825bbc322758eca0733" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 41109 } }
I20260812 01:22:17.650110 19056 ts_tablet_manager.cc:1428] T a593d4f08adc482aac259684573a4907 P 55139ee42d004825bbc322758eca0733: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 01:22:17.650856 19723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:17.646600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.131:35249 (local address 127.0.0.1:40600)
0812 01:22:17.646812 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0812 01:22:17.646825 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 01:22:17.646881 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:22:17.647399 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:22:17.647400 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:22:17.647402 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:22:17.647456 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:22:17.647458 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:22:17.648486 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:22:17.648488 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:22:17.649568 (+ 1080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:22:17.649580 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:22:17.650578 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:22:17.650579 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:22:17.650653 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:22:17.650657 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 01:22:17.650724 (+ 67us) client_negotiation.cc:240] Negotiation successful
0812 01:22:17.650769 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":117,"threads_started":1}
I20260812 01:22:17.651054 19725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:17.646916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40600 (local address 127.24.0.131:35249)
0812 01:22:17.647231 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0812 01:22:17.647236 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 01:22:17.647247 (+ 11us) server_negotiation.cc:407] Connection header received
0812 01:22:17.647291 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:22:17.647293 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:22:17.647307 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:22:17.647401 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:22:17.647571 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:17.648400 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:17.649678 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:17.650486 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:17.650565 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:22:17.650629 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 01:22:17.650958 (+ 329us) server_negotiation.cc:299] Negotiation successful
0812 01:22:17.650996 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":102,"threads_started":1}
I20260812 01:22:17.651856 19601 tablet_service.cc:1467] Processing CreateTablet for tablet a593d4f08adc482aac259684573a4907 (DEFAULT_TABLE table=test-table [id=19c40fb0bebb47a396928d8b2e2a00d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:22:17.652328 19601 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a593d4f08adc482aac259684573a4907. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:22:17.652776 19277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:17.646476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.130:39345 (local address 127.0.0.1:44470)
0812 01:22:17.646571 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0812 01:22:17.646582 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 01:22:17.646628 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:22:17.648962 (+ 2334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:22:17.648963 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:22:17.648965 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:22:17.648999 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:22:17.649001 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:22:17.650272 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:22:17.650273 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:22:17.651229 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:22:17.651232 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:22:17.652508 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:22:17.652509 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:22:17.652595 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:22:17.652599 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 01:22:17.652660 (+ 61us) client_negotiation.cc:240] Negotiation successful
0812 01:22:17.652706 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77}
I20260812 01:22:17.652796 19722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:17.646563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44470 (local address 127.24.0.130:39345)
0812 01:22:17.648766 (+ 2203us) server_negotiation.cc:206] Beginning negotiation
0812 01:22:17.648771 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 01:22:17.648783 (+ 12us) server_negotiation.cc:407] Connection header received
0812 01:22:17.648832 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:22:17.648835 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:22:17.648853 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:22:17.648955 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:22:17.649061 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:17.650153 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:17.651340 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:17.652348 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:17.652480 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:22:17.652550 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 01:22:17.652681 (+ 131us) server_negotiation.cc:299] Negotiation successful
0812 01:22:17.652720 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2157,"thread_start_us":139,"threads_started":1}
I20260812 01:22:17.653602 19392 tablet_service.cc:1467] Processing CreateTablet for tablet a593d4f08adc482aac259684573a4907 (DEFAULT_TABLE table=test-table [id=19c40fb0bebb47a396928d8b2e2a00d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:22:17.653905 19392 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a593d4f08adc482aac259684573a4907. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:22:17.655901 19727 tablet_bootstrap.cc:492] T a593d4f08adc482aac259684573a4907 P 2fb3a91da6ad4a2390d98ba13a53ed06: Bootstrap starting.
I20260812 01:22:17.656724 19727 tablet_bootstrap.cc:654] T a593d4f08adc482aac259684573a4907 P 2fb3a91da6ad4a2390d98ba13a53ed06: Neither blocks nor log segments found. Creating new log.
I20260812 01:22:17.657409 19727 log.cc:824] T a593d4f08adc482aac259684573a4907 P 2fb3a91da6ad4a2390d98ba13a53ed06: Log is configured to *not* fsync() on all Append() calls
I20260812 01:22:17.657491 19728 tablet_bootstrap.cc:492] T a593d4f08adc482aac259684573a4907 P 6cf4f241cc73458ea35718b385f9207f: Bootstrap starting.
I20260812 01:22:17.658126 19728 tablet_bootstrap.cc:654] T a593d4f08adc482aac259684573a4907 P 6cf4f241cc73458ea35718b385f9207f: Neither blocks nor log segments found. Creating new log.
I20260812 01:22:17.658340 19727 tablet_bootstrap.cc:492] T a593d4f08adc482aac259684573a4907 P 2fb3a91da6ad4a2390d98ba13a53ed06: No bootstrap required, opened a new log
I20260812 01:22:17.658424 19727 ts_tablet_manager.cc:1397] T a593d4f08adc482aac259684573a4907 P 2fb3a91da6ad4a2390d98ba13a53ed06: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 01:22:17.658694 19728 log.cc:824] T a593d4f08adc482aac259684573a4907 P 6cf4f241cc73458ea35718b385f9207f: Log is configured to *not* fsync() on all Append() calls
I20260812 01:22:17.659439 19728 tablet_bootstrap.cc:492] T a593d4f08adc482aac259684573a4907 P 6cf4f241cc73458ea35718b385f9207f: No bootstrap required, opened a new log
I20260812 01:22:17.659495 19728 ts_tablet_manager.cc:1397] T a593d4f08adc482aac259684573a4907 P 6cf4f241cc73458ea35718b385f9207f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 01:22:17.659936 19727 raft_consensus.cc:348] T a593d4f08adc482aac259684573a4907 P 2fb3a91da6ad4a2390d98ba13a53ed06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cf4f241cc73458ea35718b385f9207f" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39345 } } peers { permanent_uuid: "2fb3a91da6ad4a2390d98ba13a53ed06" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 35249 } } peers { permanent_uuid: "55139ee42d004825bbc322758eca0733" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 41109 } }
I20260812 01:22:17.660127 19727 raft_consensus.cc:374] T a593d4f08adc482aac259684573a4907 P 2fb3a91da6ad4a2390d98ba13a53ed06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:22:17.660179 19727 raft_consensus.cc:729] T a593d4f08adc482aac259684573a4907 P 2fb3a91da6ad4a2390d98ba13a53ed06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fb3a91da6ad4a2390d98ba13a53ed06, State: Initialized, Role: FOLLOWER
I20260812 01:22:17.660308 19727 consensus_queue.cc:260] T a593d4f08adc482aac259684573a4907 P 2fb3a91da6ad4a2390d98ba13a53ed06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cf4f241cc73458ea35718b385f9207f" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39345 } } peers { permanent_uuid: "2fb3a91da6ad4a2390d98ba13a53ed06" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 35249 } } peers { permanent_uuid: "55139ee42d004825bbc322758eca0733" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 41109 } }
I20260812 01:22:17.660588 19727 ts_tablet_manager.cc:1428] T a593d4f08adc482aac259684573a4907 P 2fb3a91da6ad4a2390d98ba13a53ed06: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 01:22:17.660648 19706 heartbeater.cc:503] Master 127.24.0.190:37371 was elected leader, sending a full tablet report...
I20260812 01:22:17.661357 19728 raft_consensus.cc:348] T a593d4f08adc482aac259684573a4907 P 6cf4f241cc73458ea35718b385f9207f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cf4f241cc73458ea35718b385f9207f" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39345 } } peers { permanent_uuid: "2fb3a91da6ad4a2390d98ba13a53ed06" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 35249 } } peers { permanent_uuid: "55139ee42d004825bbc322758eca0733" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 41109 } }
I20260812 01:22:17.661520 19728 raft_consensus.cc:374] T a593d4f08adc482aac259684573a4907 P 6cf4f241cc73458ea35718b385f9207f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:22:17.661556 19728 raft_consensus.cc:729] T a593d4f08adc482aac259684573a4907 P 6cf4f241cc73458ea35718b385f9207f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cf4f241cc73458ea35718b385f9207f, State: Initialized, Role: FOLLOWER
I20260812 01:22:17.661671 19728 consensus_queue.cc:260] T a593d4f08adc482aac259684573a4907 P 6cf4f241cc73458ea35718b385f9207f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cf4f241cc73458ea35718b385f9207f" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39345 } } peers { permanent_uuid: "2fb3a91da6ad4a2390d98ba13a53ed06" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 35249 } } peers { permanent_uuid: "55139ee42d004825bbc322758eca0733" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 41109 } }
I20260812 01:22:17.661947 19728 ts_tablet_manager.cc:1428] T a593d4f08adc482aac259684573a4907 P 6cf4f241cc73458ea35718b385f9207f: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260812 01:22:17.661995 19492 heartbeater.cc:503] Master 127.24.0.190:37371 was elected leader, sending a full tablet report...
I20260812 01:22:17.705245 19734 raft_consensus.cc:482] T a593d4f08adc482aac259684573a4907 P 6cf4f241cc73458ea35718b385f9207f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 01:22:17.705369 19734 raft_consensus.cc:504] T a593d4f08adc482aac259684573a4907 P 6cf4f241cc73458ea35718b385f9207f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cf4f241cc73458ea35718b385f9207f" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39345 } } peers { permanent_uuid: "2fb3a91da6ad4a2390d98ba13a53ed06" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 35249 } } peers { permanent_uuid: "55139ee42d004825bbc322758eca0733" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 41109 } }
I20260812 01:22:17.705744 19734 leader_election.cc:288] T a593d4f08adc482aac259684573a4907 P 6cf4f241cc73458ea35718b385f9207f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2fb3a91da6ad4a2390d98ba13a53ed06 (127.24.0.131:35249), 55139ee42d004825bbc322758eca0733 (127.24.0.129:41109)
I20260812 01:22:17.709031 19497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:17.705964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.131:35249 (local address 127.24.0.130:41237)
0812 01:22:17.706022 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0812 01:22:17.706036 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 01:22:17.706088 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:22:17.706290 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:22:17.706291 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:22:17.706294 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:22:17.706373 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:22:17.706375 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:22:17.707088 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:22:17.707089 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:22:17.708185 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:22:17.708188 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:22:17.708757 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:22:17.708758 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:22:17.708844 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:22:17.708848 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 01:22:17.708911 (+ 63us) client_negotiation.cc:240] Negotiation successful
0812 01:22:17.708949 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260812 01:22:17.709022 19725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:17.706085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.130:41237 (local address 127.24.0.131:35249)
0812 01:22:17.706152 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0812 01:22:17.706154 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:22:17.706161 (+ 7us) server_negotiation.cc:407] Connection header received
0812 01:22:17.706186 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:22:17.706187 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:22:17.706195 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:22:17.706268 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:22:17.706463 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:17.707011 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:17.708273 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:17.708662 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:17.708748 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:22:17.708758 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 01:22:17.708917 (+ 159us) server_negotiation.cc:299] Negotiation successful
0812 01:22:17.708955 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 01:22:17.709122 19280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:17.706046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.130:49389 (local address 127.24.0.129:41109)
0812 01:22:17.706106 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0812 01:22:17.706108 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:22:17.706350 (+ 242us) server_negotiation.cc:407] Connection header received
0812 01:22:17.706387 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:22:17.706388 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:22:17.706399 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:22:17.706459 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:22:17.706635 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:17.707111 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:17.708287 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:17.708749 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:17.708848 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:22:17.708857 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 01:22:17.709012 (+ 155us) server_negotiation.cc:299] Negotiation successful
0812 01:22:17.709054 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 01:22:17.709348 19737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:17.705966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.129:41109 (local address 127.24.0.130:49389)
0812 01:22:17.706282 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0812 01:22:17.706298 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 01:22:17.706348 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:22:17.706481 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:22:17.706482 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:22:17.706484 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:22:17.706562 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:22:17.706564 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:22:17.707216 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:22:17.707217 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:22:17.708190 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:22:17.708193 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:22:17.708850 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:22:17.708851 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:22:17.708934 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:22:17.708937 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 01:22:17.709005 (+ 68us) client_negotiation.cc:240] Negotiation successful
0812 01:22:17.709046 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"mutex_wait_us":106,"thread_start_us":156,"threads_started":1}
I20260812 01:22:17.709362 19638 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a593d4f08adc482aac259684573a4907" candidate_uuid: "6cf4f241cc73458ea35718b385f9207f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fb3a91da6ad4a2390d98ba13a53ed06" is_pre_election: true
I20260812 01:22:17.709546 19638 raft_consensus.cc:2454] T a593d4f08adc482aac259684573a4907 P 2fb3a91da6ad4a2390d98ba13a53ed06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6cf4f241cc73458ea35718b385f9207f in term 0.
I20260812 01:22:17.709682 19176 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a593d4f08adc482aac259684573a4907" candidate_uuid: "6cf4f241cc73458ea35718b385f9207f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55139ee42d004825bbc322758eca0733" is_pre_election: true
I20260812 01:22:17.709808 19311 leader_election.cc:302] T a593d4f08adc482aac259684573a4907 P 6cf4f241cc73458ea35718b385f9207f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2fb3a91da6ad4a2390d98ba13a53ed06, 6cf4f241cc73458ea35718b385f9207f; no voters:
I20260812 01:22:17.709851 19176 raft_consensus.cc:2454] T a593d4f08adc482aac259684573a4907 P 55139ee42d004825bbc322758eca0733 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6cf4f241cc73458ea35718b385f9207f in term 0.
I20260812 01:22:17.709954 19734 raft_consensus.cc:2781] T a593d4f08adc482aac259684573a4907 P 6cf4f241cc73458ea35718b385f9207f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 01:22:17.710009 19734 raft_consensus.cc:482] T a593d4f08adc482aac259684573a4907 P 6cf4f241cc73458ea35718b385f9207f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 01:22:17.710039 19734 raft_consensus.cc:3037] T a593d4f08adc482aac259684573a4907 P 6cf4f241cc73458ea35718b385f9207f [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:22:17.710790 19734 raft_consensus.cc:504] T a593d4f08adc482aac259684573a4907 P 6cf4f241cc73458ea35718b385f9207f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cf4f241cc73458ea35718b385f9207f" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39345 } } peers { permanent_uuid: "2fb3a91da6ad4a2390d98ba13a53ed06" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 35249 } } peers { permanent_uuid: "55139ee42d004825bbc322758eca0733" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 41109 } }
I20260812 01:22:17.710947 19734 leader_election.cc:288] T a593d4f08adc482aac259684573a4907 P 6cf4f241cc73458ea35718b385f9207f [CANDIDATE]: Term 1 election: Requested vote from peers 2fb3a91da6ad4a2390d98ba13a53ed06 (127.24.0.131:35249), 55139ee42d004825bbc322758eca0733 (127.24.0.129:41109)
I20260812 01:22:17.711105 19638 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a593d4f08adc482aac259684573a4907" candidate_uuid: "6cf4f241cc73458ea35718b385f9207f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fb3a91da6ad4a2390d98ba13a53ed06"
I20260812 01:22:17.711102 19176 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a593d4f08adc482aac259684573a4907" candidate_uuid: "6cf4f241cc73458ea35718b385f9207f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55139ee42d004825bbc322758eca0733"
I20260812 01:22:17.711189 19176 raft_consensus.cc:3037] T a593d4f08adc482aac259684573a4907 P 55139ee42d004825bbc322758eca0733 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:22:17.711192 19638 raft_consensus.cc:3037] T a593d4f08adc482aac259684573a4907 P 2fb3a91da6ad4a2390d98ba13a53ed06 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:22:17.711944 19638 raft_consensus.cc:2454] T a593d4f08adc482aac259684573a4907 P 2fb3a91da6ad4a2390d98ba13a53ed06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6cf4f241cc73458ea35718b385f9207f in term 1.
I20260812 01:22:17.712157 19311 leader_election.cc:302] T a593d4f08adc482aac259684573a4907 P 6cf4f241cc73458ea35718b385f9207f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2fb3a91da6ad4a2390d98ba13a53ed06, 6cf4f241cc73458ea35718b385f9207f; no voters:
I20260812 01:22:17.712338 19734 raft_consensus.cc:2781] T a593d4f08adc482aac259684573a4907 P 6cf4f241cc73458ea35718b385f9207f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:22:17.712513 19734 raft_consensus.cc:686] T a593d4f08adc482aac259684573a4907 P 6cf4f241cc73458ea35718b385f9207f [term 1 LEADER]: Becoming Leader. State: Replica: 6cf4f241cc73458ea35718b385f9207f, State: Running, Role: LEADER
I20260812 01:22:17.712652 19176 raft_consensus.cc:2454] T a593d4f08adc482aac259684573a4907 P 55139ee42d004825bbc322758eca0733 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6cf4f241cc73458ea35718b385f9207f in term 1.
I20260812 01:22:17.712649 19734 consensus_queue.cc:237] T a593d4f08adc482aac259684573a4907 P 6cf4f241cc73458ea35718b385f9207f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cf4f241cc73458ea35718b385f9207f" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39345 } } peers { permanent_uuid: "2fb3a91da6ad4a2390d98ba13a53ed06" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 35249 } } peers { permanent_uuid: "55139ee42d004825bbc322758eca0733" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 41109 } }
I20260812 01:22:17.713617 18917 catalog_manager.cc:5495] T a593d4f08adc482aac259684573a4907 P 6cf4f241cc73458ea35718b385f9207f reported cstate change: term changed from 0 to 1, leader changed from <none> to 6cf4f241cc73458ea35718b385f9207f (127.24.0.130). New cstate: current_term: 1 leader_uuid: "6cf4f241cc73458ea35718b385f9207f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cf4f241cc73458ea35718b385f9207f" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39345 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2fb3a91da6ad4a2390d98ba13a53ed06" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 35249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "55139ee42d004825bbc322758eca0733" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 41109 } health_report { overall_health: UNKNOWN } } }
I20260812 01:22:17.726665 19722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:17.724393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44480 (local address 127.24.0.130:39345)
0812 01:22:17.724461 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0812 01:22:17.724463 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:22:17.724474 (+ 11us) server_negotiation.cc:407] Connection header received
0812 01:22:17.724507 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:22:17.724507 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:22:17.724526 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:22:17.724592 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:22:17.724701 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:17.725296 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:17.725985 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:17.726193 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:17.726249 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:22:17.726375 (+ 126us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:22:17.726471 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:22:17.726532 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 01:22:17.726572 (+ 40us) server_negotiation.cc:299] Negotiation successful
0812 01:22:17.726589 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 01:22:17.728534 19638 raft_consensus.cc:1261] T a593d4f08adc482aac259684573a4907 P 2fb3a91da6ad4a2390d98ba13a53ed06 [term 1 FOLLOWER]: Refusing update from remote peer 6cf4f241cc73458ea35718b385f9207f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:22:17.728557 19176 raft_consensus.cc:1261] T a593d4f08adc482aac259684573a4907 P 55139ee42d004825bbc322758eca0733 [term 1 FOLLOWER]: Refusing update from remote peer 6cf4f241cc73458ea35718b385f9207f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:22:17.728933 19734 consensus_queue.cc:1035] T a593d4f08adc482aac259684573a4907 P 6cf4f241cc73458ea35718b385f9207f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2fb3a91da6ad4a2390d98ba13a53ed06" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 35249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:22:17.729110 19739 consensus_queue.cc:1035] T a593d4f08adc482aac259684573a4907 P 6cf4f241cc73458ea35718b385f9207f [LEADER]: Connected to new peer: Peer: permanent_uuid: "55139ee42d004825bbc322758eca0733" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 41109 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:22:17.731153 19742 mvcc.cc:205] Tried to move back new op lower bound from 7317494733733167104 to 7317494733671813120. Current Snapshot: MvccSnapshot[applied={T|T < 7317494733733167104}]
I20260812 01:22:17.731560 19744 mvcc.cc:205] Tried to move back new op lower bound from 7317494733733167104 to 7317494733671813120. Current Snapshot: MvccSnapshot[applied={T|T < 7317494733733167104}]
I20260812 01:22:17.731778 19743 mvcc.cc:205] Tried to move back new op lower bound from 7317494733733167104 to 7317494733671813120. Current Snapshot: MvccSnapshot[applied={T|T < 7317494733733167104}]
I20260812 01:22:17.734129 18918 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57642:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:22:17.734227 18918 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57642:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:22:17.735167 18918 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 30cc471efc684639a2d06046514aaf95: Sending DeleteTablet for 3 replicas of tablet a593d4f08adc482aac259684573a4907
I20260812 01:22:17.735517 19392 tablet_service.cc:1514] Processing DeleteTablet for tablet a593d4f08adc482aac259684573a4907 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:22:17 UTC) from {username='kudu', principal='kudu/127.24.0.190@KRBTEST.COM'} at 127.0.0.1:44470
I20260812 01:22:17.735535 19601 tablet_service.cc:1514] Processing DeleteTablet for tablet a593d4f08adc482aac259684573a4907 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:22:17 UTC) from {username='kudu', principal='kudu/127.24.0.190@KRBTEST.COM'} at 127.0.0.1:40600
I20260812 01:22:17.735782 19754 tablet_replica.cc:330] T a593d4f08adc482aac259684573a4907 P 2fb3a91da6ad4a2390d98ba13a53ed06: stopping tablet replica
I20260812 01:22:17.735855 19142 tablet_service.cc:1514] Processing DeleteTablet for tablet a593d4f08adc482aac259684573a4907 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:22:17 UTC) from {username='kudu', principal='kudu/127.24.0.190@KRBTEST.COM'} at 127.0.0.1:35204
I20260812 01:22:17.735883 19754 raft_consensus.cc:2229] T a593d4f08adc482aac259684573a4907 P 2fb3a91da6ad4a2390d98ba13a53ed06 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 01:22:17.735942 19281 tablet_replica.cc:330] T a593d4f08adc482aac259684573a4907 P 55139ee42d004825bbc322758eca0733: stopping tablet replica
I20260812 01:22:17.735980 19281 raft_consensus.cc:2229] T a593d4f08adc482aac259684573a4907 P 55139ee42d004825bbc322758eca0733 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 01:22:17.735965 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 18978
I20260812 01:22:17.735983 19754 raft_consensus.cc:2258] T a593d4f08adc482aac259684573a4907 P 2fb3a91da6ad4a2390d98ba13a53ed06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:22:17.736034 19281 raft_consensus.cc:2258] T a593d4f08adc482aac259684573a4907 P 55139ee42d004825bbc322758eca0733 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:22:17.746986 19753 tablet_replica.cc:330] T a593d4f08adc482aac259684573a4907 P 6cf4f241cc73458ea35718b385f9207f: stopping tablet replica
I20260812 01:22:17.747138 19754 ts_tablet_manager.cc:1905] T a593d4f08adc482aac259684573a4907 P 2fb3a91da6ad4a2390d98ba13a53ed06: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:22:17.747292 19753 raft_consensus.cc:2229] T a593d4f08adc482aac259684573a4907 P 6cf4f241cc73458ea35718b385f9207f [term 1 LEADER]: Raft consensus shutting down.
I20260812 01:22:17.747514 19753 raft_consensus.cc:2258] T a593d4f08adc482aac259684573a4907 P 6cf4f241cc73458ea35718b385f9207f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:22:17.748050 19753 ts_tablet_manager.cc:1905] T a593d4f08adc482aac259684573a4907 P 6cf4f241cc73458ea35718b385f9207f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:22:17.749215 19754 ts_tablet_manager.cc:1918] T a593d4f08adc482aac259684573a4907 P 2fb3a91da6ad4a2390d98ba13a53ed06: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 01:22:17.749310 19754 log.cc:1196] T a593d4f08adc482aac259684573a4907 P 2fb3a91da6ad4a2390d98ba13a53ed06: Deleting WAL directory at /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-2/wal/wals/a593d4f08adc482aac259684573a4907
I20260812 01:22:17.749639 19754 ts_tablet_manager.cc:1939] T a593d4f08adc482aac259684573a4907 P 2fb3a91da6ad4a2390d98ba13a53ed06: Deleting consensus metadata
I20260812 01:22:17.750168 18894 catalog_manager.cc:4841] TS 2fb3a91da6ad4a2390d98ba13a53ed06 (127.24.0.131:35249): tablet a593d4f08adc482aac259684573a4907 (table test-table [id=19c40fb0bebb47a396928d8b2e2a00d6]) successfully deleted
I20260812 01:22:17.750234 19753 ts_tablet_manager.cc:1918] T a593d4f08adc482aac259684573a4907 P 6cf4f241cc73458ea35718b385f9207f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 01:22:17.750382 19753 log.cc:1196] T a593d4f08adc482aac259684573a4907 P 6cf4f241cc73458ea35718b385f9207f: Deleting WAL directory at /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-1/wal/wals/a593d4f08adc482aac259684573a4907
I20260812 01:22:17.750659 19753 ts_tablet_manager.cc:1939] T a593d4f08adc482aac259684573a4907 P 6cf4f241cc73458ea35718b385f9207f: Deleting consensus metadata
I20260812 01:22:17.751085 18892 catalog_manager.cc:4841] TS 6cf4f241cc73458ea35718b385f9207f (127.24.0.130:39345): tablet a593d4f08adc482aac259684573a4907 (table test-table [id=19c40fb0bebb47a396928d8b2e2a00d6]) successfully deleted
W20260812 01:22:17.754849 18891 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.0.129:41109 (error 108)
I20260812 01:22:17.755182 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 19273
I20260812 01:22:17.760673 19277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:17.755295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.129:41109 (local address 127.0.0.1:35212)
0812 01:22:17.760555 (+ 5260us) negotiation.cc:107] Waiting for socket to connect
0812 01:22:17.760625 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.129:41109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":5234}
W20260812 01:22:17.760810 18891 catalog_manager.cc:4584] TS 55139ee42d004825bbc322758eca0733 (127.24.0.129:41109): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a593d4f08adc482aac259684573a4907: Network error: Client connection negotiation failed: client connection to 127.24.0.129:41109: connect: Connection refused (error 111)
I20260812 01:22:17.768479 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 19498
I20260812 01:22:17.780300 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 18854
I20260812 01:22:37.789451 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.190:37371
--webserver_interface=127.24.0.190
--webserver_port=43471
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab
--principal=kudu/127.24.0.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:40363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.0.190:37371
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:22:37.874979 19757 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:22:37.875185 19757 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:22:37.875206 19757 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:22:37.876304 19757 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260812 01:22:37.876338 19757 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260812 01:22:37.876351 19757 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260812 01:22:37.876362 19757 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:22:37.876375 19757 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:22:37.876387 19757 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:22:37.876399 19757 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:22:37.876411 19757 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:22:37.877871 19757 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:40363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.24.0.190:37371
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab
--principal=kudu/127.24.0.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.190:37371
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.0.190
--webserver_port=43471
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19757
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:22:37.878065 19757 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:22:37.878232 19757 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 01:22:37.883301 19757 server_base.cc:1034] running on GCE node
W20260812 01:22:37.883272 19763 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:22:37.883299 19766 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:22:37.883284 19775 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:22:37.883807 19757 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:22:37.884207 19757 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:22:37.885383 19757 hybrid_clock.cc:648] HybridClock initialized: now 1786497757885365 us; error 41 us; skew 500 ppm
Aug 12 01:22:37 dist-test-slave-t3q3 krb5kdc[17894](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497757, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:22:37.887974 19757 init.cc:377] Logged in from keytab as kudu/127.24.0.190@KRBTEST.COM (short username kudu)
I20260812 01:22:37.888447 19757 webserver.cc:460] Webserver started at http://127.24.0.190:43471/ using document root <none> and password file <none>
I20260812 01:22:37.888682 19757 fs_manager.cc:359] Metadata directory not provided
I20260812 01:22:37.888731 19757 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:22:37.890095 19757 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 01:22:37.890833 19785 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:22:37.891026 19757 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 01:22:37.891126 19757 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/master-0/wal
uuid: "30cc471efc684639a2d06046514aaf95"
format_stamp: "Formatted at 2026-08-12 01:21:56 on dist-test-slave-t3q3"
server_key: "6c3c8d0e32baad54521b8a63f0346eae"
server_key_iv: "2784481ffa70883c7439aee8d9d1c5c8"
server_key_version: "encryptionkey@0"
I20260812 01:22:37.891476 19757 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:22:37.914062 19757 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:22:37.916344 19757 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:22:37.916476 19757 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:22:37.920966 19757 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.190:37371
I20260812 01:22:37.921049 19869 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.190:37371 every 8 connection(s)
I20260812 01:22:37.921511 19757 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
I20260812 01:22:37.923089 19871 sys_catalog.cc:263] Verifying existing consensus state
I20260812 01:22:37.923810 19871 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30cc471efc684639a2d06046514aaf95: Bootstrap starting.
I20260812 01:22:37.926240 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 19757
I20260812 01:22:37.926570 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.0.129:41109
--local_ip_for_outbound_sockets=127.24.0.129
--tserver_master_addrs=127.24.0.190:37371
--webserver_port=36545
--webserver_interface=127.24.0.129
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.129.keytab
--principal=kudu/127.24.0.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:40363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 01:22:37.933683 19871 log.cc:824] T 00000000000000000000000000000000 P 30cc471efc684639a2d06046514aaf95: Log is configured to *not* fsync() on all Append() calls
I20260812 01:22:37.936003 19871 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30cc471efc684639a2d06046514aaf95: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 01:22:37.936381 19871 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30cc471efc684639a2d06046514aaf95: Bootstrap complete.
I20260812 01:22:37.938695 19871 raft_consensus.cc:348] T 00000000000000000000000000000000 P 30cc471efc684639a2d06046514aaf95 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30cc471efc684639a2d06046514aaf95" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 37371 } }
I20260812 01:22:37.938951 19871 raft_consensus.cc:729] T 00000000000000000000000000000000 P 30cc471efc684639a2d06046514aaf95 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30cc471efc684639a2d06046514aaf95, State: Initialized, Role: FOLLOWER
I20260812 01:22:37.939173 19871 consensus_queue.cc:260] T 00000000000000000000000000000000 P 30cc471efc684639a2d06046514aaf95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30cc471efc684639a2d06046514aaf95" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 37371 } }
I20260812 01:22:37.939277 19871 raft_consensus.cc:388] T 00000000000000000000000000000000 P 30cc471efc684639a2d06046514aaf95 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:22:37.939333 19871 raft_consensus.cc:482] T 00000000000000000000000000000000 P 30cc471efc684639a2d06046514aaf95 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:22:37.939383 19871 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 30cc471efc684639a2d06046514aaf95 [term 2 FOLLOWER]: Advancing to term 3
I20260812 01:22:37.940460 19871 raft_consensus.cc:504] T 00000000000000000000000000000000 P 30cc471efc684639a2d06046514aaf95 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30cc471efc684639a2d06046514aaf95" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 37371 } }
I20260812 01:22:37.940583 19871 leader_election.cc:302] T 00000000000000000000000000000000 P 30cc471efc684639a2d06046514aaf95 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 30cc471efc684639a2d06046514aaf95; no voters:
I20260812 01:22:37.940796 19871 leader_election.cc:288] T 00000000000000000000000000000000 P 30cc471efc684639a2d06046514aaf95 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260812 01:22:37.940833 19878 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 30cc471efc684639a2d06046514aaf95 [term 3 FOLLOWER]: Leader election won for term 3
I20260812 01:22:37.940959 19878 raft_consensus.cc:686] T 00000000000000000000000000000000 P 30cc471efc684639a2d06046514aaf95 [term 3 LEADER]: Becoming Leader. State: Replica: 30cc471efc684639a2d06046514aaf95, State: Running, Role: LEADER
I20260812 01:22:37.941092 19871 sys_catalog.cc:564] T 00000000000000000000000000000000 P 30cc471efc684639a2d06046514aaf95 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:22:37.941076 19878 consensus_queue.cc:237] T 00000000000000000000000000000000 P 30cc471efc684639a2d06046514aaf95 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30cc471efc684639a2d06046514aaf95" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 37371 } }
I20260812 01:22:37.941452 19880 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30cc471efc684639a2d06046514aaf95 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "30cc471efc684639a2d06046514aaf95" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30cc471efc684639a2d06046514aaf95" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 37371 } } }
I20260812 01:22:37.941550 19880 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30cc471efc684639a2d06046514aaf95 [sys.catalog]: This master's current role is: LEADER
I20260812 01:22:37.941782 19882 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30cc471efc684639a2d06046514aaf95 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 30cc471efc684639a2d06046514aaf95. Latest consensus state: current_term: 3 leader_uuid: "30cc471efc684639a2d06046514aaf95" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30cc471efc684639a2d06046514aaf95" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 37371 } } }
I20260812 01:22:37.941893 19882 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30cc471efc684639a2d06046514aaf95 [sys.catalog]: This master's current role is: LEADER
I20260812 01:22:37.942036 19890 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:22:37.942869 19890 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:22:37.943125 19890 catalog_manager.cc:1259] Loaded cluster ID: d4417e3d8d03430ab68a6c014449d70a
I20260812 01:22:37.943219 19890 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 01:22:37.944451 19890 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:22:37.944653 19890 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 30cc471efc684639a2d06046514aaf95: Loaded TSK: 0
I20260812 01:22:37.944706 19890 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 30cc471efc684639a2d06046514aaf95: Loaded TSK: 1
F20260812 01:22:37.944933 19890 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f2dac11d9ad google::LogMessage::Fail() at ??:0
@ 0x7f2dac12088a google::LogMessage::SendToLog() at ??:0
@ 0x7f2dac11d4de google::LogMessage::Flush() at ??:0
@ 0x7f2dac11ea19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f2dae88a104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f2dae887d41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f2db40771cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f2db407748e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f2db4077594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f2db403dc48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f2db4047ad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f2dadfeb0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f2dadfe0954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f2daee446db start_thread at ??:0
@ 0x7f2daac1b61f clone at ??:0
W20260812 01:22:38.029073 19874 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:22:38.029310 19874 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:22:38.029350 19874 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:22:38.030507 19874 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:22:38.030577 19874 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:22:38.030624 19874 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.0.129
I20260812 01:22:38.032400 19874 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:40363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.129.keytab
--principal=kudu/127.24.0.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.0.129:41109
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.0.129
--webserver_port=36545
--tserver_master_addrs=127.24.0.190:37371
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19874
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.0.129
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:22:38.032682 19874 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:22:38.032958 19874 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:22:38.038272 19938 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:22:38.038385 19874 server_base.cc:1034] running on GCE node
W20260812 01:22:38.038290 19943 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:22:38.038508 19937 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:22:38.038750 19874 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:22:38.039073 19874 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:22:38.040242 19874 hybrid_clock.cc:648] HybridClock initialized: now 1786497758040223 us; error 32 us; skew 500 ppm
Aug 12 01:22:38 dist-test-slave-t3q3 krb5kdc[17894](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497758, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:22:38.043109 19874 init.cc:377] Logged in from keytab as kudu/127.24.0.129@KRBTEST.COM (short username kudu)
I20260812 01:22:38.043690 19874 webserver.cc:460] Webserver started at http://127.24.0.129:36545/ using document root <none> and password file <none>
I20260812 01:22:38.043959 19874 fs_manager.cc:359] Metadata directory not provided
I20260812 01:22:38.044034 19874 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:22:38.045450 19874 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:22:38.046190 19953 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:22:38.046449 19874 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:22:38.046552 19874 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-0/wal
uuid: "55139ee42d004825bbc322758eca0733"
format_stamp: "Formatted at 2026-08-12 01:21:56 on dist-test-slave-t3q3"
server_key: "e505175fce3d46c439ffb1acc925b3aa"
server_key_iv: "5824cdee799dda6087b788590ad2a7ad"
server_key_version: "encryptionkey@0"
I20260812 01:22:38.046998 19874 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:22:38.063889 19874 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:22:38.066484 19874 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:22:38.066654 19874 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:22:38.066916 19874 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:22:38.067435 19963 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 01:22:38.069135 19874 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 01:22:38.069198 19874 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260812 01:22:38.069231 19874 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 01:22:38.070113 19874 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 01:22:38.070165 19874 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260812 01:22:38.070258 19963 tablet_bootstrap.cc:492] T a593d4f08adc482aac259684573a4907 P 55139ee42d004825bbc322758eca0733: Bootstrap starting.
I20260812 01:22:38.077694 19874 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.129:41109
I20260812 01:22:38.078135 20138 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.129:41109 every 8 connection(s)
I20260812 01:22:38.078296 19874 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-0/data/info.pb
I20260812 01:22:38.082140 19963 log.cc:824] T a593d4f08adc482aac259684573a4907 P 55139ee42d004825bbc322758eca0733: Log is configured to *not* fsync() on all Append() calls
I20260812 01:22:38.083629 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 19874
I20260812 01:22:38.083964 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.0.130:39345
--local_ip_for_outbound_sockets=127.24.0.130
--tserver_master_addrs=127.24.0.190:37371
--webserver_port=41971
--webserver_interface=127.24.0.130
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.130.keytab
--principal=kudu/127.24.0.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:40363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 01:22:38.084933 19963 tablet_bootstrap.cc:492] T a593d4f08adc482aac259684573a4907 P 55139ee42d004825bbc322758eca0733: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 01:22:38.085282 19963 tablet_bootstrap.cc:492] T a593d4f08adc482aac259684573a4907 P 55139ee42d004825bbc322758eca0733: Bootstrap complete.
I20260812 01:22:38.086460 19963 ts_tablet_manager.cc:1397] T a593d4f08adc482aac259684573a4907 P 55139ee42d004825bbc322758eca0733: Time spent bootstrapping tablet: real 0.016s user 0.010s sys 0.004s
I20260812 01:22:38.087559 19963 raft_consensus.cc:348] T a593d4f08adc482aac259684573a4907 P 55139ee42d004825bbc322758eca0733 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cf4f241cc73458ea35718b385f9207f" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39345 } } peers { permanent_uuid: "2fb3a91da6ad4a2390d98ba13a53ed06" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 35249 } } peers { permanent_uuid: "55139ee42d004825bbc322758eca0733" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 41109 } }
I20260812 01:22:38.087832 19963 raft_consensus.cc:729] T a593d4f08adc482aac259684573a4907 P 55139ee42d004825bbc322758eca0733 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55139ee42d004825bbc322758eca0733, State: Initialized, Role: FOLLOWER
I20260812 01:22:38.088001 19963 consensus_queue.cc:260] T a593d4f08adc482aac259684573a4907 P 55139ee42d004825bbc322758eca0733 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cf4f241cc73458ea35718b385f9207f" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39345 } } peers { permanent_uuid: "2fb3a91da6ad4a2390d98ba13a53ed06" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 35249 } } peers { permanent_uuid: "55139ee42d004825bbc322758eca0733" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 41109 } }
I20260812 01:22:38.088346 19963 ts_tablet_manager.cc:1428] T a593d4f08adc482aac259684573a4907 P 55139ee42d004825bbc322758eca0733: Time spent starting tablet: real 0.002s user 0.004s sys 0.000s
W20260812 01:22:38.156404 20142 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:22:38.079088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:37371 (local address 127.24.0.129:41233)
0812 01:22:38.079328 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0812 01:22:38.079387 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0812 01:22:38.080250 (+ 863us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:22:38.156203 (+ 75953us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:37371: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":102,"threads_started":1}
W20260812 01:22:38.156740 20139 heartbeater.cc:644] Failed to heartbeat to 127.24.0.190:37371 (0 consecutive failures): Network error: Failed to ping master at 127.24.0.190:37371: Client connection negotiation failed: client connection to 127.24.0.190:37371: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 01:22:38.157099 20142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:38.156976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:37371 (local address 127.24.0.129:47157)
0812 01:22:38.157033 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0812 01:22:38.157081 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:37371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260812 01:22:38.157528 20142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:38.157440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:37371 (local address 127.24.0.129:42225)
0812 01:22:38.157488 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0812 01:22:38.157513 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:37371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260812 01:22:38.157660 20139 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 01:22:38.174468 20146 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:22:38.174670 20146 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:22:38.174695 20146 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:22:38.175844 20146 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:22:38.175889 20146 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:22:38.175916 20146 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.0.130
I20260812 01:22:38.177433 20146 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:40363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.130.keytab
--principal=kudu/127.24.0.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.0.130:39345
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.0.130
--webserver_port=41971
--tserver_master_addrs=127.24.0.190:37371
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20146
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.0.130
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:22:38.177685 20146 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:22:38.177953 20146 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:22:38.183048 20157 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:22:38.183043 20159 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:22:38.183043 20156 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:22:38.183343 20146 server_base.cc:1034] running on GCE node
I20260812 01:22:38.183524 20146 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:22:38.183736 20146 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:22:38.184880 20146 hybrid_clock.cc:648] HybridClock initialized: now 1786497758184865 us; error 29 us; skew 500 ppm
Aug 12 01:22:38 dist-test-slave-t3q3 krb5kdc[17894](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497758, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:22:38.187350 20146 init.cc:377] Logged in from keytab as kudu/127.24.0.130@KRBTEST.COM (short username kudu)
I20260812 01:22:38.187733 20146 webserver.cc:460] Webserver started at http://127.24.0.130:41971/ using document root <none> and password file <none>
I20260812 01:22:38.187925 20146 fs_manager.cc:359] Metadata directory not provided
I20260812 01:22:38.187966 20146 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:22:38.189272 20146 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:22:38.189850 20167 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:22:38.190083 20146 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:22:38.190200 20146 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-1/wal
uuid: "6cf4f241cc73458ea35718b385f9207f"
format_stamp: "Formatted at 2026-08-12 01:21:56 on dist-test-slave-t3q3"
server_key: "7961aa150737e2fcea0b71ba23418d1e"
server_key_iv: "6d79c2a710d80a3af872ef54a5d36a45"
server_key_version: "encryptionkey@0"
I20260812 01:22:38.190529 20146 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:22:38.211105 20146 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:22:38.213112 20146 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:22:38.213297 20146 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:22:38.213543 20146 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:22:38.213871 20146 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:22:38.213924 20146 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:22:38.213959 20146 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:22:38.213987 20146 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:22:38.219916 20146 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.130:39345
I20260812 01:22:38.220016 20342 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.130:39345 every 8 connection(s)
I20260812 01:22:38.220378 20146 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-1/data/info.pb
I20260812 01:22:38.221539 20345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:38.221090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:37371 (local address 127.24.0.130:60157)
0812 01:22:38.221318 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0812 01:22:38.221394 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:37371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":116,"threads_started":1}
I20260812 01:22:38.221796 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 20146
W20260812 01:22:38.222044 20343 heartbeater.cc:644] Failed to heartbeat to 127.24.0.190:37371 (0 consecutive failures): Network error: Failed to ping master at 127.24.0.190:37371: Client connection negotiation failed: client connection to 127.24.0.190:37371: connect: Connection refused (error 111)
I20260812 01:22:38.222074 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.0.131:35249
--local_ip_for_outbound_sockets=127.24.0.131
--tserver_master_addrs=127.24.0.190:37371
--webserver_port=33579
--webserver_interface=127.24.0.131
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.131.keytab
--principal=kudu/127.24.0.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:40363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 01:22:38.222390 20345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:38.222269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:37371 (local address 127.24.0.130:60753)
0812 01:22:38.222325 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0812 01:22:38.222358 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:37371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260812 01:22:38.222757 20345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:38.222652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:37371 (local address 127.24.0.130:49163)
0812 01:22:38.222699 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0812 01:22:38.222729 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:37371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260812 01:22:38.222937 20343 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 01:22:38.306433 20347 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:22:38.306689 20347 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:22:38.306732 20347 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:22:38.307897 20347 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:22:38.307965 20347 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:22:38.308012 20347 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.0.131
I20260812 01:22:38.309513 20347 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:40363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.131.keytab
--principal=kudu/127.24.0.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.0.131:35249
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.0.131
--webserver_port=33579
--tserver_master_addrs=127.24.0.190:37371
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20347
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.0.131
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:22:38.309803 20347 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:22:38.310036 20347 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:22:38.315239 20355 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:22:38.315228 20356 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:22:38.315232 20359 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:22:38.315543 20347 server_base.cc:1034] running on GCE node
I20260812 01:22:38.315758 20347 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:22:38.316040 20347 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:22:38.317270 20347 hybrid_clock.cc:648] HybridClock initialized: now 1786497758317256 us; error 30 us; skew 500 ppm
Aug 12 01:22:38 dist-test-slave-t3q3 krb5kdc[17894](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497758, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:22:38.320171 20347 init.cc:377] Logged in from keytab as kudu/127.24.0.131@KRBTEST.COM (short username kudu)
I20260812 01:22:38.320619 20347 webserver.cc:460] Webserver started at http://127.24.0.131:33579/ using document root <none> and password file <none>
I20260812 01:22:38.320873 20347 fs_manager.cc:359] Metadata directory not provided
I20260812 01:22:38.320945 20347 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:22:38.322361 20347 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:22:38.322983 20374 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:22:38.323207 20347 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:22:38.323323 20347 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-2/wal
uuid: "2fb3a91da6ad4a2390d98ba13a53ed06"
format_stamp: "Formatted at 2026-08-12 01:21:56 on dist-test-slave-t3q3"
server_key: "2984abd5ee12c68ca949e6f9be6adc19"
server_key_iv: "4acc62bdf1abc43efab71c1f259d601a"
server_key_version: "encryptionkey@0"
I20260812 01:22:38.323679 20347 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:22:38.356963 20347 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:22:38.359508 20347 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:22:38.359684 20347 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:22:38.359953 20347 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:22:38.360471 20347 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:22:38.360548 20347 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:22:38.360590 20347 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:22:38.360620 20347 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:22:38.366518 20347 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.131:35249
I20260812 01:22:38.366576 20549 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.131:35249 every 8 connection(s)
I20260812 01:22:38.366912 20347 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786497534195564-24578-0/minicluster-data/ts-2/data/info.pb
I20260812 01:22:38.367964 20553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:38.367588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:37371 (local address 127.24.0.131:44747)
0812 01:22:38.367750 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0812 01:22:38.367824 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:37371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":73,"threads_started":1}
I20260812 01:22:38.368402 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 20347
W20260812 01:22:38.368413 20551 heartbeater.cc:644] Failed to heartbeat to 127.24.0.190:37371 (0 consecutive failures): Network error: Failed to ping master at 127.24.0.190:37371: Client connection negotiation failed: client connection to 127.24.0.190:37371: connect: Connection refused (error 111)
I20260812 01:22:38.368706 20553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:38.368621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:37371 (local address 127.24.0.131:34957)
0812 01:22:38.368657 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0812 01:22:38.368686 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:37371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260812 01:22:38.369133 20553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:38.369034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:37371 (local address 127.24.0.131:48791)
0812 01:22:38.369079 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0812 01:22:38.369105 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:37371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260812 01:22:38.369240 20551 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 01:22:39.158310 20557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:39.158053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:37371 (local address 127.24.0.129:53207)
0812 01:22:39.158200 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0812 01:22:39.158244 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:37371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":75,"threads_started":1}
I20260812 01:22:39.223564 20558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:39.223296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:37371 (local address 127.24.0.130:43697)
0812 01:22:39.223450 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0812 01:22:39.223494 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:37371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":67,"threads_started":1}
I20260812 01:22:39.369783 20560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:39.369561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:37371 (local address 127.24.0.131:40971)
0812 01:22:39.369697 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0812 01:22:39.369734 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:37371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":69,"threads_started":1}
I20260812 01:22:39.678727 20561 raft_consensus.cc:482] T a593d4f08adc482aac259684573a4907 P 55139ee42d004825bbc322758eca0733 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 01:22:39.678989 20561 raft_consensus.cc:504] T a593d4f08adc482aac259684573a4907 P 55139ee42d004825bbc322758eca0733 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cf4f241cc73458ea35718b385f9207f" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39345 } } peers { permanent_uuid: "2fb3a91da6ad4a2390d98ba13a53ed06" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 35249 } } peers { permanent_uuid: "55139ee42d004825bbc322758eca0733" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 41109 } }
I20260812 01:22:39.679545 20561 leader_election.cc:288] T a593d4f08adc482aac259684573a4907 P 55139ee42d004825bbc322758eca0733 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6cf4f241cc73458ea35718b385f9207f (127.24.0.130:39345), 2fb3a91da6ad4a2390d98ba13a53ed06 (127.24.0.131:35249)
Aug 12 01:22:39 dist-test-slave-t3q3 krb5kdc[17894](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497758, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.129@KRBTEST.COM for kudu/127.24.0.130@KRBTEST.COM
Aug 12 01:22:39 dist-test-slave-t3q3 krb5kdc[17894](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497758, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.129@KRBTEST.COM for kudu/127.24.0.131@KRBTEST.COM
I20260812 01:22:39.707281 20573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:39.680022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.129:59907 (local address 127.24.0.130:39345)
0812 01:22:39.680321 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0812 01:22:39.680345 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0812 01:22:39.680431 (+ 86us) server_negotiation.cc:407] Connection header received
0812 01:22:39.681537 (+ 1106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:22:39.681543 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:22:39.681672 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:22:39.681990 (+ 318us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:22:39.684160 (+ 2170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:39.686603 (+ 2443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:39.689493 (+ 2890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:39.690731 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:39.690993 (+ 262us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:22:39.698928 (+ 7935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:22:39.698943 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:22:39.698943 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:22:39.698986 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:22:39.703016 (+ 4030us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:22:39.704171 (+ 1155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:22:39.704179 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:22:39.704180 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:22:39.704432 (+ 252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:22:39.705101 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:22:39.705110 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:22:39.705110 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:22:39.705779 (+ 669us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:22:39.705949 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0812 01:22:39.706805 (+ 856us) server_negotiation.cc:299] Negotiation successful
0812 01:22:39.707114 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":244,"thread_start_us":183,"threads_started":1}
I20260812 01:22:39.707275 20572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:39.680022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.129:43999 (local address 127.24.0.131:35249)
0812 01:22:39.680360 (+ 338us) server_negotiation.cc:206] Beginning negotiation
0812 01:22:39.680376 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 01:22:39.680439 (+ 63us) server_negotiation.cc:407] Connection header received
0812 01:22:39.681537 (+ 1098us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:22:39.681543 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:22:39.681672 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:22:39.681990 (+ 318us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:22:39.684160 (+ 2170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:39.686602 (+ 2442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:39.689493 (+ 2891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:39.690731 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:39.690993 (+ 262us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:22:39.699579 (+ 8586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:22:39.699593 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:22:39.699593 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:22:39.699627 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:22:39.703417 (+ 3790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:22:39.704299 (+ 882us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:22:39.704311 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:22:39.704311 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:22:39.704501 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:22:39.705089 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:22:39.705090 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:22:39.705090 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:22:39.705779 (+ 689us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:22:39.705949 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0812 01:22:39.706805 (+ 856us) server_negotiation.cc:299] Negotiation successful
0812 01:22:39.707085 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282,"thread_start_us":124,"threads_started":1}
I20260812 01:22:39.707355 20563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:39.679786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.130:39345 (local address 127.24.0.129:59907)
0812 01:22:39.680097 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0812 01:22:39.680116 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 01:22:39.680228 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:22:39.682096 (+ 1868us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:22:39.682100 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:22:39.683189 (+ 1089us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:22:39.683864 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:22:39.683873 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:22:39.686866 (+ 2993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:22:39.686867 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:22:39.689348 (+ 2481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:22:39.689351 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:22:39.690912 (+ 1561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:22:39.690914 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:22:39.691156 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:22:39.695239 (+ 4083us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:22:39.695256 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:22:39.698640 (+ 3384us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:22:39.703186 (+ 4546us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:22:39.703193 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:22:39.703196 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:22:39.703984 (+ 788us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:22:39.704566 (+ 582us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:22:39.704568 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:22:39.704568 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:22:39.704949 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:22:39.705916 (+ 967us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:22:39.705924 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:22:39.706343 (+ 419us) client_negotiation.cc:769] Sending connection context
0812 01:22:39.706652 (+ 309us) client_negotiation.cc:240] Negotiation successful
0812 01:22:39.707094 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229,"mutex_wait_us":133,"thread_start_us":148,"threads_started":1}
I20260812 01:22:39.707648 20562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:39.679786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.131:35249 (local address 127.24.0.129:43999)
0812 01:22:39.680050 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0812 01:22:39.680117 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0812 01:22:39.680228 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:22:39.682096 (+ 1868us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:22:39.682100 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:22:39.683189 (+ 1089us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:22:39.683864 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:22:39.683873 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:22:39.686845 (+ 2972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:22:39.686846 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:22:39.689348 (+ 2502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:22:39.689351 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:22:39.690910 (+ 1559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:22:39.690912 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:22:39.691156 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:22:39.695239 (+ 4083us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:22:39.695256 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:22:39.699328 (+ 4072us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:22:39.703562 (+ 4234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:22:39.703572 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:22:39.703572 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:22:39.704162 (+ 590us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:22:39.704592 (+ 430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:22:39.704593 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:22:39.704593 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:22:39.704950 (+ 357us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:22:39.705916 (+ 966us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:22:39.705924 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:22:39.706343 (+ 419us) client_negotiation.cc:769] Sending connection context
0812 01:22:39.706652 (+ 309us) client_negotiation.cc:240] Negotiation successful
0812 01:22:39.707187 (+ 535us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"mutex_wait_us":399,"thread_start_us":150,"threads_started":1}
W20260812 01:22:39.709009 19956 leader_election.cc:341] T a593d4f08adc482aac259684573a4907 P 55139ee42d004825bbc322758eca0733 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6cf4f241cc73458ea35718b385f9207f (127.24.0.130:39345): Not found: Tablet not found: a593d4f08adc482aac259684573a4907
W20260812 01:22:39.709437 19959 leader_election.cc:341] T a593d4f08adc482aac259684573a4907 P 55139ee42d004825bbc322758eca0733 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2fb3a91da6ad4a2390d98ba13a53ed06 (127.24.0.131:35249): Not found: Tablet not found: a593d4f08adc482aac259684573a4907
I20260812 01:22:39.709563 19959 leader_election.cc:302] T a593d4f08adc482aac259684573a4907 P 55139ee42d004825bbc322758eca0733 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 55139ee42d004825bbc322758eca0733; no voters: 2fb3a91da6ad4a2390d98ba13a53ed06, 6cf4f241cc73458ea35718b385f9207f
I20260812 01:22:39.709945 20561 raft_consensus.cc:2726] T a593d4f08adc482aac259684573a4907 P 55139ee42d004825bbc322758eca0733 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 01:22:40.159161 20562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:40.158945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:37371 (local address 127.24.0.129:47717)
0812 01:22:40.159028 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0812 01:22:40.159107 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:37371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260812 01:22:40.224493 20578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:40.224118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:37371 (local address 127.24.0.130:46391)
0812 01:22:40.224382 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0812 01:22:40.224437 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:37371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":171,"threads_started":1}
I20260812 01:22:40.370810 20580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:40.370528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:37371 (local address 127.24.0.131:60693)
0812 01:22:40.370697 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0812 01:22:40.370749 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:37371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":94,"threads_started":1}
I20260812 01:22:41.160151 20583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:41.159786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:37371 (local address 127.24.0.129:55419)
0812 01:22:41.159971 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0812 01:22:41.160027 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:37371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":113,"threads_started":1}
I20260812 01:22:41.225268 20584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:41.225062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:37371 (local address 127.24.0.130:38943)
0812 01:22:41.225187 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0812 01:22:41.225226 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:37371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":65,"threads_started":1}
I20260812 01:22:41.371630 20585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:41.371412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:37371 (local address 127.24.0.131:59319)
0812 01:22:41.371544 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0812 01:22:41.371590 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:37371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":66,"threads_started":1}
I20260812 01:22:41.526750 20586 raft_consensus.cc:482] T a593d4f08adc482aac259684573a4907 P 55139ee42d004825bbc322758eca0733 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 01:22:41.526845 20586 raft_consensus.cc:504] T a593d4f08adc482aac259684573a4907 P 55139ee42d004825bbc322758eca0733 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cf4f241cc73458ea35718b385f9207f" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39345 } } peers { permanent_uuid: "2fb3a91da6ad4a2390d98ba13a53ed06" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 35249 } } peers { permanent_uuid: "55139ee42d004825bbc322758eca0733" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 41109 } }
I20260812 01:22:41.527014 20586 leader_election.cc:288] T a593d4f08adc482aac259684573a4907 P 55139ee42d004825bbc322758eca0733 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6cf4f241cc73458ea35718b385f9207f (127.24.0.130:39345), 2fb3a91da6ad4a2390d98ba13a53ed06 (127.24.0.131:35249)
W20260812 01:22:41.527509 19959 leader_election.cc:341] T a593d4f08adc482aac259684573a4907 P 55139ee42d004825bbc322758eca0733 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2fb3a91da6ad4a2390d98ba13a53ed06 (127.24.0.131:35249): Not found: Tablet not found: a593d4f08adc482aac259684573a4907
W20260812 01:22:41.527590 19956 leader_election.cc:341] T a593d4f08adc482aac259684573a4907 P 55139ee42d004825bbc322758eca0733 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6cf4f241cc73458ea35718b385f9207f (127.24.0.130:39345): Not found: Tablet not found: a593d4f08adc482aac259684573a4907
I20260812 01:22:41.527670 19959 leader_election.cc:302] T a593d4f08adc482aac259684573a4907 P 55139ee42d004825bbc322758eca0733 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 55139ee42d004825bbc322758eca0733; no voters: 2fb3a91da6ad4a2390d98ba13a53ed06, 6cf4f241cc73458ea35718b385f9207f
I20260812 01:22:41.527770 20586 raft_consensus.cc:2726] T a593d4f08adc482aac259684573a4907 P 55139ee42d004825bbc322758eca0733 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 01:22:42.160915 20587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:42.160659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:37371 (local address 127.24.0.129:34111)
0812 01:22:42.160808 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0812 01:22:42.160867 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:37371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":84,"threads_started":1}
I20260812 01:22:42.226032 20588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:42.225757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:37371 (local address 127.24.0.130:34107)
0812 01:22:42.225934 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0812 01:22:42.225970 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:37371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":75,"threads_started":1}
I20260812 01:22:42.372464 20589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:42.372230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:37371 (local address 127.24.0.131:41889)
0812 01:22:42.372376 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0812 01:22:42.372417 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:37371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":67,"threads_started":1}
I20260812 01:22:43.161880 20590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:43.161605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:37371 (local address 127.24.0.129:59711)
0812 01:22:43.161773 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0812 01:22:43.161823 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:37371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":69,"threads_started":1}
I20260812 01:22:43.226766 20591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:43.226556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:37371 (local address 127.24.0.130:55385)
0812 01:22:43.226684 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0812 01:22:43.226721 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:37371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":68,"threads_started":1}
I20260812 01:22:43.373271 20592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:43.373007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:37371 (local address 127.24.0.131:59177)
0812 01:22:43.373186 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0812 01:22:43.373223 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:37371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":70,"threads_started":1}
I20260812 01:22:43.604471 20593 raft_consensus.cc:482] T a593d4f08adc482aac259684573a4907 P 55139ee42d004825bbc322758eca0733 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 01:22:43.604710 20593 raft_consensus.cc:504] T a593d4f08adc482aac259684573a4907 P 55139ee42d004825bbc322758eca0733 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cf4f241cc73458ea35718b385f9207f" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39345 } } peers { permanent_uuid: "2fb3a91da6ad4a2390d98ba13a53ed06" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 35249 } } peers { permanent_uuid: "55139ee42d004825bbc322758eca0733" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 41109 } }
I20260812 01:22:43.604944 20593 leader_election.cc:288] T a593d4f08adc482aac259684573a4907 P 55139ee42d004825bbc322758eca0733 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6cf4f241cc73458ea35718b385f9207f (127.24.0.130:39345), 2fb3a91da6ad4a2390d98ba13a53ed06 (127.24.0.131:35249)
W20260812 01:22:43.605336 19956 leader_election.cc:341] T a593d4f08adc482aac259684573a4907 P 55139ee42d004825bbc322758eca0733 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6cf4f241cc73458ea35718b385f9207f (127.24.0.130:39345): Not found: Tablet not found: a593d4f08adc482aac259684573a4907
W20260812 01:22:43.605410 19959 leader_election.cc:341] T a593d4f08adc482aac259684573a4907 P 55139ee42d004825bbc322758eca0733 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2fb3a91da6ad4a2390d98ba13a53ed06 (127.24.0.131:35249): Not found: Tablet not found: a593d4f08adc482aac259684573a4907
I20260812 01:22:43.605448 19959 leader_election.cc:302] T a593d4f08adc482aac259684573a4907 P 55139ee42d004825bbc322758eca0733 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 55139ee42d004825bbc322758eca0733; no voters: 2fb3a91da6ad4a2390d98ba13a53ed06, 6cf4f241cc73458ea35718b385f9207f
I20260812 01:22:43.605551 20593 raft_consensus.cc:2726] T a593d4f08adc482aac259684573a4907 P 55139ee42d004825bbc322758eca0733 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 01:22:44.162704 20594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:44.162477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:37371 (local address 127.24.0.129:46755)
0812 01:22:44.162612 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0812 01:22:44.162653 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:37371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":66,"threads_started":1}
I20260812 01:22:44.227453 20596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:44.227239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:37371 (local address 127.24.0.130:41523)
0812 01:22:44.227373 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0812 01:22:44.227410 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:37371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":61,"threads_started":1}
I20260812 01:22:44.374011 20598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:44.373791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:37371 (local address 127.24.0.131:49017)
0812 01:22:44.373930 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0812 01:22:44.373968 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:37371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":66,"threads_started":1}
I20260812 01:22:45.163468 20599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:45.163200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:37371 (local address 127.24.0.129:49817)
0812 01:22:45.163383 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0812 01:22:45.163422 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:37371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":103,"threads_started":1}
I20260812 01:22:45.228286 20600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:45.227971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:37371 (local address 127.24.0.130:37987)
0812 01:22:45.228188 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0812 01:22:45.228232 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:37371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":66,"threads_started":1}
I20260812 01:22:45.374686 20602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:45.374478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:37371 (local address 127.24.0.131:54191)
0812 01:22:45.374603 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0812 01:22:45.374640 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:37371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":64,"threads_started":1}
I20260812 01:22:45.822582 20604 raft_consensus.cc:482] T a593d4f08adc482aac259684573a4907 P 55139ee42d004825bbc322758eca0733 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 01:22:45.822681 20604 raft_consensus.cc:504] T a593d4f08adc482aac259684573a4907 P 55139ee42d004825bbc322758eca0733 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cf4f241cc73458ea35718b385f9207f" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39345 } } peers { permanent_uuid: "2fb3a91da6ad4a2390d98ba13a53ed06" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 35249 } } peers { permanent_uuid: "55139ee42d004825bbc322758eca0733" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 41109 } }
I20260812 01:22:45.822901 20604 leader_election.cc:288] T a593d4f08adc482aac259684573a4907 P 55139ee42d004825bbc322758eca0733 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6cf4f241cc73458ea35718b385f9207f (127.24.0.130:39345), 2fb3a91da6ad4a2390d98ba13a53ed06 (127.24.0.131:35249)
W20260812 01:22:45.823451 19956 leader_election.cc:341] T a593d4f08adc482aac259684573a4907 P 55139ee42d004825bbc322758eca0733 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6cf4f241cc73458ea35718b385f9207f (127.24.0.130:39345): Not found: Tablet not found: a593d4f08adc482aac259684573a4907
W20260812 01:22:45.823589 19959 leader_election.cc:341] T a593d4f08adc482aac259684573a4907 P 55139ee42d004825bbc322758eca0733 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2fb3a91da6ad4a2390d98ba13a53ed06 (127.24.0.131:35249): Not found: Tablet not found: a593d4f08adc482aac259684573a4907
I20260812 01:22:45.823666 19959 leader_election.cc:302] T a593d4f08adc482aac259684573a4907 P 55139ee42d004825bbc322758eca0733 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 55139ee42d004825bbc322758eca0733; no voters: 2fb3a91da6ad4a2390d98ba13a53ed06, 6cf4f241cc73458ea35718b385f9207f
I20260812 01:22:45.823764 20604 raft_consensus.cc:2726] T a593d4f08adc482aac259684573a4907 P 55139ee42d004825bbc322758eca0733 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 01:22:46.164276 20605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:46.164009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:37371 (local address 127.24.0.129:59483)
0812 01:22:46.164184 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0812 01:22:46.164226 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:37371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":84,"threads_started":1}
I20260812 01:22:46.229056 20606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:46.228787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:37371 (local address 127.24.0.130:45047)
0812 01:22:46.228963 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0812 01:22:46.229006 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:37371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":91,"threads_started":1}
I20260812 01:22:46.375478 20607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:46.375254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:37371 (local address 127.24.0.131:48839)
0812 01:22:46.375393 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0812 01:22:46.375433 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:37371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":59,"threads_started":1}
I20260812 01:22:47.165143 20614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:47.164786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:37371 (local address 127.24.0.129:52521)
0812 01:22:47.165060 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0812 01:22:47.165096 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:37371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":210,"threads_started":1}
I20260812 01:22:47.229802 20615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:47.229568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:37371 (local address 127.24.0.130:37397)
0812 01:22:47.229718 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0812 01:22:47.229755 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:37371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":82,"threads_started":1}
I20260812 01:22:47.376407 20617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:47.376151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:37371 (local address 127.24.0.131:50685)
0812 01:22:47.376311 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0812 01:22:47.376359 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:37371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":88,"threads_started":1}
I20260812 01:22:48.165894 20618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:48.165660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:37371 (local address 127.24.0.129:45893)
0812 01:22:48.165810 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0812 01:22:48.165847 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:37371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":66,"threads_started":1}
I20260812 01:22:48.230635 20619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:48.230379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:37371 (local address 127.24.0.130:48647)
0812 01:22:48.230548 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0812 01:22:48.230588 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:37371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":86,"threads_started":1}
I20260812 01:22:48.377147 20621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:48.376886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:37371 (local address 127.24.0.131:58713)
0812 01:22:48.377065 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0812 01:22:48.377103 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:37371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":88,"threads_started":1}
I20260812 01:22:49.089725 20622 raft_consensus.cc:482] T a593d4f08adc482aac259684573a4907 P 55139ee42d004825bbc322758eca0733 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 01:22:49.089820 20622 raft_consensus.cc:504] T a593d4f08adc482aac259684573a4907 P 55139ee42d004825bbc322758eca0733 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cf4f241cc73458ea35718b385f9207f" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39345 } } peers { permanent_uuid: "2fb3a91da6ad4a2390d98ba13a53ed06" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 35249 } } peers { permanent_uuid: "55139ee42d004825bbc322758eca0733" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 41109 } }
I20260812 01:22:49.090049 20622 leader_election.cc:288] T a593d4f08adc482aac259684573a4907 P 55139ee42d004825bbc322758eca0733 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6cf4f241cc73458ea35718b385f9207f (127.24.0.130:39345), 2fb3a91da6ad4a2390d98ba13a53ed06 (127.24.0.131:35249)
W20260812 01:22:49.090538 19959 leader_election.cc:341] T a593d4f08adc482aac259684573a4907 P 55139ee42d004825bbc322758eca0733 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2fb3a91da6ad4a2390d98ba13a53ed06 (127.24.0.131:35249): Not found: Tablet not found: a593d4f08adc482aac259684573a4907
W20260812 01:22:49.090651 19956 leader_election.cc:341] T a593d4f08adc482aac259684573a4907 P 55139ee42d004825bbc322758eca0733 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6cf4f241cc73458ea35718b385f9207f (127.24.0.130:39345): Not found: Tablet not found: a593d4f08adc482aac259684573a4907
I20260812 01:22:49.090689 19956 leader_election.cc:302] T a593d4f08adc482aac259684573a4907 P 55139ee42d004825bbc322758eca0733 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 55139ee42d004825bbc322758eca0733; no voters: 2fb3a91da6ad4a2390d98ba13a53ed06, 6cf4f241cc73458ea35718b385f9207f
I20260812 01:22:49.090785 20622 raft_consensus.cc:2726] T a593d4f08adc482aac259684573a4907 P 55139ee42d004825bbc322758eca0733 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 01:22:49.166700 20624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:49.166446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:37371 (local address 127.24.0.129:48789)
0812 01:22:49.166612 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0812 01:22:49.166652 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:37371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":77,"threads_started":1}
I20260812 01:22:49.231310 20625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:49.231102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:37371 (local address 127.24.0.130:40069)
0812 01:22:49.231228 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0812 01:22:49.231268 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:37371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":82,"threads_started":1}
I20260812 01:22:49.377957 20626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:49.377680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:37371 (local address 127.24.0.131:42815)
0812 01:22:49.377863 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0812 01:22:49.377909 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:37371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":90,"threads_started":1}
I20260812 01:22:50.167542 20627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:50.167283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:37371 (local address 127.24.0.129:32869)
0812 01:22:50.167449 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0812 01:22:50.167489 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:37371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":87,"threads_started":1}
I20260812 01:22:50.232134 20628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:50.231852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:37371 (local address 127.24.0.130:59759)
0812 01:22:50.231984 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0812 01:22:50.232027 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:37371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":64,"threads_started":1}
I20260812 01:22:50.378800 20629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:50.378537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:37371 (local address 127.24.0.131:38517)
0812 01:22:50.378707 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0812 01:22:50.378753 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:37371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":93,"threads_started":1}
I20260812 01:22:50.899313 20630 raft_consensus.cc:482] T a593d4f08adc482aac259684573a4907 P 55139ee42d004825bbc322758eca0733 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 01:22:50.899417 20630 raft_consensus.cc:504] T a593d4f08adc482aac259684573a4907 P 55139ee42d004825bbc322758eca0733 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cf4f241cc73458ea35718b385f9207f" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39345 } } peers { permanent_uuid: "2fb3a91da6ad4a2390d98ba13a53ed06" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 35249 } } peers { permanent_uuid: "55139ee42d004825bbc322758eca0733" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 41109 } }
I20260812 01:22:50.899680 20630 leader_election.cc:288] T a593d4f08adc482aac259684573a4907 P 55139ee42d004825bbc322758eca0733 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6cf4f241cc73458ea35718b385f9207f (127.24.0.130:39345), 2fb3a91da6ad4a2390d98ba13a53ed06 (127.24.0.131:35249)
W20260812 01:22:50.900161 19956 leader_election.cc:341] T a593d4f08adc482aac259684573a4907 P 55139ee42d004825bbc322758eca0733 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6cf4f241cc73458ea35718b385f9207f (127.24.0.130:39345): Not found: Tablet not found: a593d4f08adc482aac259684573a4907
W20260812 01:22:50.900233 19959 leader_election.cc:341] T a593d4f08adc482aac259684573a4907 P 55139ee42d004825bbc322758eca0733 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2fb3a91da6ad4a2390d98ba13a53ed06 (127.24.0.131:35249): Not found: Tablet not found: a593d4f08adc482aac259684573a4907
I20260812 01:22:50.900274 19959 leader_election.cc:302] T a593d4f08adc482aac259684573a4907 P 55139ee42d004825bbc322758eca0733 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 55139ee42d004825bbc322758eca0733; no voters: 2fb3a91da6ad4a2390d98ba13a53ed06, 6cf4f241cc73458ea35718b385f9207f
I20260812 01:22:50.900355 20630 raft_consensus.cc:2726] T a593d4f08adc482aac259684573a4907 P 55139ee42d004825bbc322758eca0733 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 01:22:51.168248 20632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:51.168005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:37371 (local address 127.24.0.129:43885)
0812 01:22:51.168163 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0812 01:22:51.168201 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:37371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":139,"threads_started":1}
I20260812 01:22:51.232905 20633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:51.232604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:37371 (local address 127.24.0.130:56975)
0812 01:22:51.232758 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0812 01:22:51.232801 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:37371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":80,"threads_started":1}
I20260812 01:22:51.379565 20634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:51.379332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:37371 (local address 127.24.0.131:54835)
0812 01:22:51.379461 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0812 01:22:51.379503 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:37371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":68,"threads_started":1}
I20260812 01:22:52.168980 20635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:52.168732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:37371 (local address 127.24.0.129:49573)
0812 01:22:52.168894 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0812 01:22:52.168931 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:37371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":70,"threads_started":1}
I20260812 01:22:52.233642 20638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:52.233424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:37371 (local address 127.24.0.130:50087)
0812 01:22:52.233557 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0812 01:22:52.233591 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:37371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":67,"threads_started":1}
I20260812 01:22:52.380390 20639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:52.380142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:37371 (local address 127.24.0.131:59863)
0812 01:22:52.380293 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0812 01:22:52.380338 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:37371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":66,"threads_started":1}
I20260812 01:22:53.169765 20640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:53.169536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:37371 (local address 127.24.0.129:37627)
0812 01:22:53.169679 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0812 01:22:53.169717 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:37371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":64,"threads_started":1}
I20260812 01:22:53.234418 20642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:53.234165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:37371 (local address 127.24.0.130:50033)
0812 01:22:53.234325 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0812 01:22:53.234372 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:37371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":87,"threads_started":1}
I20260812 01:22:53.377883 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 19874
I20260812 01:22:53.384217 20644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:53.383853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:37371 (local address 127.24.0.131:57651)
0812 01:22:53.384112 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0812 01:22:53.384150 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.190:37371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":59,"threads_started":1}
I20260812 01:22:53.392336 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 20146
I20260812 01:22:53.402055 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 20347
2026-08-12T01:22:53Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58724 ms)
[----------] 30 tests from SecurityITest (239225 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786497534195564-24578-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 01:22:53 dist-test-slave-t3q3 krb5kdc[20652](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 01:22:53 dist-test-slave-t3q3 krb5kdc[20652](info): set up 2 sockets
Aug 12 01:22:53 dist-test-slave-t3q3 krb5kdc[20652](info): commencing operation
krb5kdc: starting...
W20260812 01:22:54.707453 24578 mini_kdc.cc:121] Time spent starting KDC: real 1.273s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786497534195564-24578-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786497534195564-24578-0/krb5kdc/test-user.keytab.
Aug 12 01:22:54 dist-test-slave-t3q3 krb5kdc[20652](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497774, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T01:22:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:22:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.0.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.0.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.0.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab.
Entry for principal kudu/127.24.0.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab.
I20260812 01:22:54.832049 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786497534195564-24578-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786497534195564-24578-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786497534195564-24578-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.190:45263
--webserver_interface=127.24.0.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab
--principal=kudu/127.24.0.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:45969
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.0.190:45263
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:22:54.914772 20673 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:22:54.915027 20673 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:22:54.915067 20673 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:22:54.916438 20673 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:22:54.916504 20673 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:22:54.916531 20673 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:22:54.916564 20673 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:22:54.916590 20673 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:22:54.918054 20673 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:45969
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786497534195564-24578-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786497534195564-24578-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.0.190:45263
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab
--principal=kudu/127.24.0.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.190:45263
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.0.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20673
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786497534195564-24578-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:22:54.918342 20673 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:22:54.918569 20673 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:22:54.923761 20681 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:22:54.923781 20673 server_base.cc:1034] running on GCE node
W20260812 01:22:54.923780 20678 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:22:54.923761 20684 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:22:54.924360 20673 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:22:54.924698 20673 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:22:54.925855 20673 hybrid_clock.cc:648] HybridClock initialized: now 1786497774925845 us; error 48 us; skew 500 ppm
Aug 12 01:22:54 dist-test-slave-t3q3 krb5kdc[20652](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497774, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:22:54.928481 20673 init.cc:377] Logged in from keytab as kudu/127.24.0.190@KRBTEST.COM (short username kudu)
I20260812 01:22:54.928941 20673 webserver.cc:460] Webserver started at http://127.24.0.190:40389/ using document root <none> and password file <none>
I20260812 01:22:54.929133 20673 fs_manager.cc:359] Metadata directory not provided
I20260812 01:22:54.929180 20673 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:22:54.929270 20673 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:22:54.930315 20673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786497534195564-24578-0/minicluster-data/master-0/data/instance:
uuid: "9842a47e153a4d138ec8dd2baf1b2cb3"
format_stamp: "Formatted at 2026-08-12 01:22:54 on dist-test-slave-t3q3"
server_key: "2c3d5cfdc28f365d0c1604b7b30dfba8"
server_key_iv: "a03b605a7ed12b279f0752132d9f7fd1"
server_key_version: "encryptionkey@0"
I20260812 01:22:54.930621 20673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786497534195564-24578-0/minicluster-data/master-0/wal/instance:
uuid: "9842a47e153a4d138ec8dd2baf1b2cb3"
format_stamp: "Formatted at 2026-08-12 01:22:54 on dist-test-slave-t3q3"
server_key: "2c3d5cfdc28f365d0c1604b7b30dfba8"
server_key_iv: "a03b605a7ed12b279f0752132d9f7fd1"
server_key_version: "encryptionkey@0"
I20260812 01:22:54.931917 20673 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 01:22:54.932734 20695 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:22:54.932957 20673 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 01:22:54.933048 20673 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786497534195564-24578-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786497534195564-24578-0/minicluster-data/master-0/wal
uuid: "9842a47e153a4d138ec8dd2baf1b2cb3"
format_stamp: "Formatted at 2026-08-12 01:22:54 on dist-test-slave-t3q3"
server_key: "2c3d5cfdc28f365d0c1604b7b30dfba8"
server_key_iv: "a03b605a7ed12b279f0752132d9f7fd1"
server_key_version: "encryptionkey@0"
I20260812 01:22:54.933166 20673 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786497534195564-24578-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786497534195564-24578-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786497534195564-24578-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:22:54.953928 20673 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:22:54.956475 20673 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:22:54.956635 20673 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:22:54.961110 20673 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.190:45263
I20260812 01:22:54.961179 20786 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.190:45263 every 8 connection(s)
I20260812 01:22:54.961550 20673 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
I20260812 01:22:54.962443 20787 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:22:54.964783 20787 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9842a47e153a4d138ec8dd2baf1b2cb3: Bootstrap starting.
I20260812 01:22:54.965386 20787 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9842a47e153a4d138ec8dd2baf1b2cb3: Neither blocks nor log segments found. Creating new log.
I20260812 01:22:54.965695 20787 log.cc:824] T 00000000000000000000000000000000 P 9842a47e153a4d138ec8dd2baf1b2cb3: Log is configured to *not* fsync() on all Append() calls
I20260812 01:22:54.966404 20787 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9842a47e153a4d138ec8dd2baf1b2cb3: No bootstrap required, opened a new log
I20260812 01:22:54.967201 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 20673
I20260812 01:22:54.967315 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786497534195564-24578-0/minicluster-data/master-0/wal/instance
I20260812 01:22:54.967559 24578 external_mini_cluster.cc:1428] Setting key 061776d7e8a51c77263c2e9d9927d182
I20260812 01:22:54.968937 20787 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9842a47e153a4d138ec8dd2baf1b2cb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9842a47e153a4d138ec8dd2baf1b2cb3" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 45263 } }
I20260812 01:22:54.969099 20787 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9842a47e153a4d138ec8dd2baf1b2cb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:22:54.969125 20787 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9842a47e153a4d138ec8dd2baf1b2cb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9842a47e153a4d138ec8dd2baf1b2cb3, State: Initialized, Role: FOLLOWER
I20260812 01:22:54.969278 20787 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9842a47e153a4d138ec8dd2baf1b2cb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9842a47e153a4d138ec8dd2baf1b2cb3" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 45263 } }
I20260812 01:22:54.969388 20787 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9842a47e153a4d138ec8dd2baf1b2cb3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:22:54.969426 20787 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9842a47e153a4d138ec8dd2baf1b2cb3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:22:54.969514 20787 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9842a47e153a4d138ec8dd2baf1b2cb3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:22:54.970278 20787 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9842a47e153a4d138ec8dd2baf1b2cb3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9842a47e153a4d138ec8dd2baf1b2cb3" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 45263 } }
I20260812 01:22:54.970371 20787 leader_election.cc:302] T 00000000000000000000000000000000 P 9842a47e153a4d138ec8dd2baf1b2cb3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9842a47e153a4d138ec8dd2baf1b2cb3; no voters:
I20260812 01:22:54.970605 20787 leader_election.cc:288] T 00000000000000000000000000000000 P 9842a47e153a4d138ec8dd2baf1b2cb3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 01:22:54.970642 20794 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9842a47e153a4d138ec8dd2baf1b2cb3 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:22:54.970757 20794 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9842a47e153a4d138ec8dd2baf1b2cb3 [term 1 LEADER]: Becoming Leader. State: Replica: 9842a47e153a4d138ec8dd2baf1b2cb3, State: Running, Role: LEADER
I20260812 01:22:54.970880 20787 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9842a47e153a4d138ec8dd2baf1b2cb3 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:22:54.970899 20794 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9842a47e153a4d138ec8dd2baf1b2cb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9842a47e153a4d138ec8dd2baf1b2cb3" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 45263 } }
I20260812 01:22:54.971221 20794 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9842a47e153a4d138ec8dd2baf1b2cb3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9842a47e153a4d138ec8dd2baf1b2cb3. Latest consensus state: current_term: 1 leader_uuid: "9842a47e153a4d138ec8dd2baf1b2cb3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9842a47e153a4d138ec8dd2baf1b2cb3" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 45263 } } }
I20260812 01:22:54.971233 20795 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9842a47e153a4d138ec8dd2baf1b2cb3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9842a47e153a4d138ec8dd2baf1b2cb3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9842a47e153a4d138ec8dd2baf1b2cb3" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 45263 } } }
I20260812 01:22:54.971426 20795 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9842a47e153a4d138ec8dd2baf1b2cb3 [sys.catalog]: This master's current role is: LEADER
I20260812 01:22:54.971342 20794 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9842a47e153a4d138ec8dd2baf1b2cb3 [sys.catalog]: This master's current role is: LEADER
I20260812 01:22:54.971720 20805 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:22:54.972290 20805 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:22:54.974579 20805 catalog_manager.cc:1347] Generated new cluster ID: 3f3f199d322b494698f6b6494c3fa18d
I20260812 01:22:54.974640 20805 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 01:22:54 dist-test-slave-t3q3 krb5kdc[20652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497774, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
I20260812 01:22:54.979321 20792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:54.968202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47462 (local address 127.24.0.190:45263)
0812 01:22:54.968380 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0812 01:22:54.968389 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 01:22:54.968430 (+ 41us) server_negotiation.cc:407] Connection header received
0812 01:22:54.969110 (+ 680us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:22:54.969120 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:22:54.969196 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:22:54.969352 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:22:54.970158 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:54.971457 (+ 1299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:54.972679 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:54.973305 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:54.973418 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:22:54.976002 (+ 2584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:22:54.976006 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:22:54.976006 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:22:54.976027 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:22:54.977800 (+ 1773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:22:54.978198 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:22:54.978201 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:22:54.978202 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:22:54.978290 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:22:54.978504 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:22:54.978505 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:22:54.978505 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:22:54.978742 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:22:54.978921 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0812 01:22:54.979032 (+ 111us) server_negotiation.cc:299] Negotiation successful
0812 01:22:54.979169 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":93,"threads_started":1}
I20260812 01:22:54.986681 20805 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:22:54.987360 20805 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:22:54.995046 20805 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9842a47e153a4d138ec8dd2baf1b2cb3: Generated new TSK 0
I20260812 01:22:54.995236 20805 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.0.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.0.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.0.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.129.keytab.
Entry for principal kudu/127.24.0.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.129.keytab.
I20260812 01:22:55.027370 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786497534195564-24578-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786497534195564-24578-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786497534195564-24578-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786497534195564-24578-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.0.129:0
--local_ip_for_outbound_sockets=127.24.0.129
--webserver_interface=127.24.0.129
--webserver_port=0
--tserver_master_addrs=127.24.0.190:45263
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.129.keytab
--principal=kudu/127.24.0.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:45969
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:22:55.112015 20829 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:22:55.112301 20829 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:22:55.112340 20829 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:22:55.113554 20829 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:22:55.113626 20829 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:22:55.113672 20829 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.0.129
I20260812 01:22:55.115137 20829 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:45969
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786497534195564-24578-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786497534195564-24578-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.129.keytab
--principal=kudu/127.24.0.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.0.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786497534195564-24578-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.0.129
--webserver_port=0
--tserver_master_addrs=127.24.0.190:45263
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20829
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.0.129
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786497534195564-24578-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:22:55.115402 20829 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:22:55.115641 20829 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:22:55.121133 20840 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:22:55.121183 20837 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:22:55.121202 20838 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:22:55.121502 20829 server_base.cc:1034] running on GCE node
I20260812 01:22:55.121698 20829 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:22:55.121945 20829 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:22:55.123143 20829 hybrid_clock.cc:648] HybridClock initialized: now 1786497775123137 us; error 42 us; skew 500 ppm
Aug 12 01:22:55 dist-test-slave-t3q3 krb5kdc[20652](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497775, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:22:55.126052 20829 init.cc:377] Logged in from keytab as kudu/127.24.0.129@KRBTEST.COM (short username kudu)
I20260812 01:22:55.126569 20829 webserver.cc:460] Webserver started at http://127.24.0.129:38173/ using document root <none> and password file <none>
I20260812 01:22:55.126807 20829 fs_manager.cc:359] Metadata directory not provided
I20260812 01:22:55.126878 20829 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:22:55.127039 20829 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:22:55.128221 20829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786497534195564-24578-0/minicluster-data/ts-0/data/instance:
uuid: "12cb390782f546369937112c759df59e"
format_stamp: "Formatted at 2026-08-12 01:22:55 on dist-test-slave-t3q3"
server_key: "8aec1ae4d3250bd389d9c2b53d251e1b"
server_key_iv: "b4c18bfe5becdde072dc4713215ca926"
server_key_version: "encryptionkey@0"
I20260812 01:22:55.128599 20829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786497534195564-24578-0/minicluster-data/ts-0/wal/instance:
uuid: "12cb390782f546369937112c759df59e"
format_stamp: "Formatted at 2026-08-12 01:22:55 on dist-test-slave-t3q3"
server_key: "8aec1ae4d3250bd389d9c2b53d251e1b"
server_key_iv: "b4c18bfe5becdde072dc4713215ca926"
server_key_version: "encryptionkey@0"
I20260812 01:22:55.130051 20829 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 01:22:55.130820 20851 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:22:55.131018 20829 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:22:55.131131 20829 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786497534195564-24578-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786497534195564-24578-0/minicluster-data/ts-0/wal
uuid: "12cb390782f546369937112c759df59e"
format_stamp: "Formatted at 2026-08-12 01:22:55 on dist-test-slave-t3q3"
server_key: "8aec1ae4d3250bd389d9c2b53d251e1b"
server_key_iv: "b4c18bfe5becdde072dc4713215ca926"
server_key_version: "encryptionkey@0"
I20260812 01:22:55.131208 20829 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786497534195564-24578-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786497534195564-24578-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786497534195564-24578-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:22:55.153085 20829 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:22:55.155247 20829 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:22:55.155416 20829 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:22:55.155668 20829 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:22:55.156011 20829 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:22:55.156070 20829 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:22:55.156138 20829 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:22:55.156167 20829 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:22:55.162176 20829 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.129:39217
I20260812 01:22:55.162235 21015 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.129:39217 every 8 connection(s)
I20260812 01:22:55.162545 20829 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786497534195564-24578-0/minicluster-data/ts-0/data/info.pb
I20260812 01:22:55.163618 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 20829
I20260812 01:22:55.163740 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786497534195564-24578-0/minicluster-data/ts-0/wal/instance
I20260812 01:22:55.163949 24578 external_mini_cluster.cc:1428] Setting key a0c630cef90f21f9a3f3e89f170f3431
Aug 12 01:22:55 dist-test-slave-t3q3 krb5kdc[20652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497775, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.129@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.24.0.130@KRBTEST.COM; defaulting to no policy
I20260812 01:22:55.174329 20792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:55.163312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.129:41985 (local address 127.24.0.190:45263)
0812 01:22:55.163403 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0812 01:22:55.163406 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:22:55.163583 (+ 177us) server_negotiation.cc:407] Connection header received
0812 01:22:55.164297 (+ 714us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:22:55.164298 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:22:55.164311 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:22:55.164377 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:22:55.165416 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:55.166188 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:55.167233 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:55.167455 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:55.167555 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:22:55.169736 (+ 2181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:22:55.169737 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:22:55.169738 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:22:55.169746 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:22:55.171404 (+ 1658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:22:55.171898 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:22:55.171899 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:22:55.171900 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:22:55.171964 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:22:55.173588 (+ 1624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:22:55.173590 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:22:55.173591 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:22:55.173713 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:22:55.173796 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 01:22:55.174065 (+ 269us) server_negotiation.cc:299] Negotiation successful
0812 01:22:55.174216 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 01:22:55.174368 21019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:55.163238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:45263 (local address 127.24.0.129:41985)
0812 01:22:55.163431 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0812 01:22:55.163484 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 01:22:55.164172 (+ 688us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:22:55.164445 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:22:55.164450 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:22:55.165120 (+ 670us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:22:55.165306 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:22:55.165311 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:22:55.166288 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:22:55.166289 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:22:55.167130 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:22:55.167133 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:22:55.167569 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:22:55.167570 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:22:55.167706 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:22:55.168262 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:22:55.168270 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:22:55.169631 (+ 1361us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:22:55.171501 (+ 1870us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:22:55.171509 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:22:55.171511 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:22:55.171824 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:22:55.173362 (+ 1538us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:22:55.173363 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:22:55.173363 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:22:55.173487 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:22:55.173797 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:22:55.173803 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:22:55.173918 (+ 115us) client_negotiation.cc:769] Sending connection context
0812 01:22:55.174002 (+ 84us) client_negotiation.cc:240] Negotiation successful
0812 01:22:55.174211 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":96,"threads_started":1}
I20260812 01:22:55.174927 21016 heartbeater.cc:347] Connected to a master server at 127.24.0.190:45263
I20260812 01:22:55.175052 21016 heartbeater.cc:464] Registering TS with master...
I20260812 01:22:55.175232 21016 heartbeater.cc:511] Master 127.24.0.190:45263 requested a full tablet report, sending...
I20260812 01:22:55.175839 20725 ts_manager.cc:194] Registered new tserver with Master: 12cb390782f546369937112c759df59e (127.24.0.129:39217)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.0.130@KRBTEST.COM" created.
I20260812 01:22:55.176544 20725 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.0.129@KRBTEST.COM'} at 127.24.0.129:41985
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.0.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.130.keytab.
Entry for principal kudu/127.24.0.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.130.keytab.
I20260812 01:22:55.190680 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786497534195564-24578-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786497534195564-24578-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786497534195564-24578-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786497534195564-24578-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.0.130:0
--local_ip_for_outbound_sockets=127.24.0.130
--webserver_interface=127.24.0.130
--webserver_port=0
--tserver_master_addrs=127.24.0.190:45263
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.130.keytab
--principal=kudu/127.24.0.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:45969
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:22:55.274256 21022 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:22:55.274454 21022 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:22:55.274478 21022 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:22:55.275627 21022 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:22:55.275673 21022 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:22:55.275700 21022 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.0.130
I20260812 01:22:55.277169 21022 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:45969
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786497534195564-24578-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786497534195564-24578-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.130.keytab
--principal=kudu/127.24.0.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.0.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786497534195564-24578-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.0.130
--webserver_port=0
--tserver_master_addrs=127.24.0.190:45263
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21022
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.0.130
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786497534195564-24578-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:22:55.277419 21022 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:22:55.277679 21022 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 01:22:55.282938 21022 server_base.cc:1034] running on GCE node
W20260812 01:22:55.283079 21035 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:22:55.283103 21030 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:22:55.283140 21033 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:22:55.283497 21022 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:22:55.283731 21022 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:22:55.284914 21022 hybrid_clock.cc:648] HybridClock initialized: now 1786497775284904 us; error 41 us; skew 500 ppm
Aug 12 01:22:55 dist-test-slave-t3q3 krb5kdc[20652](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497775, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:22:55.287438 21022 init.cc:377] Logged in from keytab as kudu/127.24.0.130@KRBTEST.COM (short username kudu)
I20260812 01:22:55.287942 21022 webserver.cc:460] Webserver started at http://127.24.0.130:45427/ using document root <none> and password file <none>
I20260812 01:22:55.288252 21022 fs_manager.cc:359] Metadata directory not provided
I20260812 01:22:55.288326 21022 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:22:55.288479 21022 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:22:55.289569 21022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786497534195564-24578-0/minicluster-data/ts-1/data/instance:
uuid: "b5a20898f61c45e8b71f8d883e190a94"
format_stamp: "Formatted at 2026-08-12 01:22:55 on dist-test-slave-t3q3"
server_key: "52d3cf61d4675c19ae8a3f428be9be9f"
server_key_iv: "54ceca3d5ecf7fff74db1abca81c0970"
server_key_version: "encryptionkey@0"
I20260812 01:22:55.289944 21022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786497534195564-24578-0/minicluster-data/ts-1/wal/instance:
uuid: "b5a20898f61c45e8b71f8d883e190a94"
format_stamp: "Formatted at 2026-08-12 01:22:55 on dist-test-slave-t3q3"
server_key: "52d3cf61d4675c19ae8a3f428be9be9f"
server_key_iv: "54ceca3d5ecf7fff74db1abca81c0970"
server_key_version: "encryptionkey@0"
I20260812 01:22:55.291302 21022 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 01:22:55.292171 21043 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:22:55.292351 21022 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:22:55.292455 21022 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786497534195564-24578-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786497534195564-24578-0/minicluster-data/ts-1/wal
uuid: "b5a20898f61c45e8b71f8d883e190a94"
format_stamp: "Formatted at 2026-08-12 01:22:55 on dist-test-slave-t3q3"
server_key: "52d3cf61d4675c19ae8a3f428be9be9f"
server_key_iv: "54ceca3d5ecf7fff74db1abca81c0970"
server_key_version: "encryptionkey@0"
I20260812 01:22:55.292553 21022 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786497534195564-24578-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786497534195564-24578-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786497534195564-24578-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:22:55.303344 21022 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:22:55.305516 21022 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:22:55.305689 21022 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:22:55.305958 21022 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:22:55.306293 21022 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:22:55.306352 21022 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:22:55.306392 21022 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:22:55.306421 21022 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:22:55.312543 21022 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.130:41905
I20260812 01:22:55.312584 21230 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.130:41905 every 8 connection(s)
I20260812 01:22:55.312914 21022 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786497534195564-24578-0/minicluster-data/ts-1/data/info.pb
I20260812 01:22:55.315824 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 21022
I20260812 01:22:55.315936 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786497534195564-24578-0/minicluster-data/ts-1/wal/instance
I20260812 01:22:55.316202 24578 external_mini_cluster.cc:1428] Setting key 78f9e54bfe4d763384a01568a1c394b5
Aug 12 01:22:55 dist-test-slave-t3q3 krb5kdc[20652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497775, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.130@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
I20260812 01:22:55.323760 21234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:55.313691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:45263 (local address 127.24.0.130:40203)
0812 01:22:55.313872 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0812 01:22:55.313908 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 01:22:55.314418 (+ 510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:22:55.314698 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:22:55.314702 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:22:55.315314 (+ 612us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:22:55.315511 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:22:55.315516 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:22:55.316666 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:22:55.316668 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:22:55.317780 (+ 1112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:22:55.317785 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:22:55.318166 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:22:55.318167 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:22:55.318314 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:22:55.318828 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:22:55.318838 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:22:55.320400 (+ 1562us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:22:55.322361 (+ 1961us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:22:55.322368 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:22:55.322370 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:22:55.322615 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:22:55.322851 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:22:55.322852 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:22:55.322852 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:22:55.322951 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:22:55.323254 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:22:55.323259 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:22:55.323373 (+ 114us) client_negotiation.cc:769] Sending connection context
0812 01:22:55.323457 (+ 84us) client_negotiation.cc:240] Negotiation successful
0812 01:22:55.323597 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":98,"threads_started":1}
I20260812 01:22:55.323750 20792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:55.313713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.130:40203 (local address 127.24.0.190:45263)
0812 01:22:55.313783 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0812 01:22:55.313786 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:22:55.313997 (+ 211us) server_negotiation.cc:407] Connection header received
0812 01:22:55.314515 (+ 518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:22:55.314516 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:22:55.314526 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:22:55.314583 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:22:55.315646 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:55.316465 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:55.317878 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:55.318076 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:55.318138 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:22:55.320497 (+ 2359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:22:55.320498 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:22:55.320499 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:22:55.320523 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:22:55.322272 (+ 1749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:22:55.322704 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:22:55.322705 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:22:55.322706 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:22:55.322774 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:22:55.323021 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:22:55.323022 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:22:55.323023 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:22:55.323182 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:22:55.323263 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 01:22:55.323519 (+ 256us) server_negotiation.cc:299] Negotiation successful
0812 01:22:55.323613 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 01:22:55.324481 21232 heartbeater.cc:347] Connected to a master server at 127.24.0.190:45263
I20260812 01:22:55.324580 21232 heartbeater.cc:464] Registering TS with master...
I20260812 01:22:55.324787 21232 heartbeater.cc:511] Master 127.24.0.190:45263 requested a full tablet report, sending...
I20260812 01:22:55.325229 20725 ts_manager.cc:194] Registered new tserver with Master: b5a20898f61c45e8b71f8d883e190a94 (127.24.0.130:41905)
I20260812 01:22:55.325614 20725 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.0.130@KRBTEST.COM'} at 127.24.0.130:40203
WARNING: no policy specified for kudu/127.24.0.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.0.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.0.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.131.keytab.
Entry for principal kudu/127.24.0.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.131.keytab.
I20260812 01:22:55.344856 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786497534195564-24578-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786497534195564-24578-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786497534195564-24578-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786497534195564-24578-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.0.131:0
--local_ip_for_outbound_sockets=127.24.0.131
--webserver_interface=127.24.0.131
--webserver_port=0
--tserver_master_addrs=127.24.0.190:45263
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.131.keytab
--principal=kudu/127.24.0.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:45969
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:22:55.430961 21243 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:22:55.431173 21243 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:22:55.431197 21243 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:22:55.432385 21243 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:22:55.432427 21243 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:22:55.432479 21243 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.0.131
I20260812 01:22:55.434083 21243 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:45969
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786497534195564-24578-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786497534195564-24578-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.131.keytab
--principal=kudu/127.24.0.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.0.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786497534195564-24578-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.0.131
--webserver_port=0
--tserver_master_addrs=127.24.0.190:45263
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21243
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.0.131
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786497534195564-24578-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:22:55.434322 21243 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:22:55.434623 21243 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:22:55.439836 21254 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:22:55.439906 21250 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:22:55.439827 21256 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:22:55.440027 21243 server_base.cc:1034] running on GCE node
I20260812 01:22:55.440340 21243 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:22:55.440534 21243 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:22:55.441701 21243 hybrid_clock.cc:648] HybridClock initialized: now 1786497775441687 us; error 28 us; skew 500 ppm
Aug 12 01:22:55 dist-test-slave-t3q3 krb5kdc[20652](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497775, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:22:55.444598 21243 init.cc:377] Logged in from keytab as kudu/127.24.0.131@KRBTEST.COM (short username kudu)
I20260812 01:22:55.445058 21243 webserver.cc:460] Webserver started at http://127.24.0.131:37779/ using document root <none> and password file <none>
I20260812 01:22:55.445264 21243 fs_manager.cc:359] Metadata directory not provided
I20260812 01:22:55.445305 21243 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:22:55.445394 21243 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:22:55.446369 21243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786497534195564-24578-0/minicluster-data/ts-2/data/instance:
uuid: "34a95d3cc0a64b9d9c1e3a6eac3d4077"
format_stamp: "Formatted at 2026-08-12 01:22:55 on dist-test-slave-t3q3"
server_key: "222754ee58f014bec2aaa95190b7cbc2"
server_key_iv: "7c502da284fdee44b49d529b2e170874"
server_key_version: "encryptionkey@0"
I20260812 01:22:55.446689 21243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786497534195564-24578-0/minicluster-data/ts-2/wal/instance:
uuid: "34a95d3cc0a64b9d9c1e3a6eac3d4077"
format_stamp: "Formatted at 2026-08-12 01:22:55 on dist-test-slave-t3q3"
server_key: "222754ee58f014bec2aaa95190b7cbc2"
server_key_iv: "7c502da284fdee44b49d529b2e170874"
server_key_version: "encryptionkey@0"
I20260812 01:22:55.448019 21243 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 01:22:55.448851 21269 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:22:55.449028 21243 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 01:22:55.449123 21243 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786497534195564-24578-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786497534195564-24578-0/minicluster-data/ts-2/wal
uuid: "34a95d3cc0a64b9d9c1e3a6eac3d4077"
format_stamp: "Formatted at 2026-08-12 01:22:55 on dist-test-slave-t3q3"
server_key: "222754ee58f014bec2aaa95190b7cbc2"
server_key_iv: "7c502da284fdee44b49d529b2e170874"
server_key_version: "encryptionkey@0"
I20260812 01:22:55.449210 21243 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786497534195564-24578-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786497534195564-24578-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786497534195564-24578-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:22:55.471144 21243 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:22:55.473480 21243 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:22:55.473655 21243 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:22:55.473903 21243 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:22:55.474221 21243 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:22:55.474277 21243 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:22:55.474323 21243 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:22:55.474345 21243 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:22:55.480720 21243 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.131:41669
I20260812 01:22:55.480772 21454 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.131:41669 every 8 connection(s)
I20260812 01:22:55.481093 21243 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786497534195564-24578-0/minicluster-data/ts-2/data/info.pb
Aug 12 01:22:55 dist-test-slave-t3q3 krb5kdc[20652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497775, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.131@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
I20260812 01:22:55.490119 20792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:55.481785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.131:47635 (local address 127.24.0.190:45263)
0812 01:22:55.481868 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0812 01:22:55.481870 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:22:55.482080 (+ 210us) server_negotiation.cc:407] Connection header received
0812 01:22:55.482567 (+ 487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:22:55.482569 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:22:55.482579 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:22:55.482635 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:22:55.483560 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:55.484050 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:55.485116 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:55.485301 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:55.485354 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:22:55.487421 (+ 2067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:22:55.487422 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:22:55.487423 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:22:55.487429 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:22:55.488797 (+ 1368us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:22:55.489196 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:22:55.489197 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:22:55.489197 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:22:55.489252 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:22:55.489516 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:22:55.489517 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:22:55.489518 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:22:55.489644 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:22:55.489706 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 01:22:55.489974 (+ 268us) server_negotiation.cc:299] Negotiation successful
0812 01:22:55.490053 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 01:22:55.490276 21460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:55.481758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:45263 (local address 127.24.0.131:47635)
0812 01:22:55.481956 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0812 01:22:55.481994 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 01:22:55.482478 (+ 484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:22:55.482688 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:22:55.482692 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:22:55.483269 (+ 577us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:22:55.483468 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:22:55.483473 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:22:55.484199 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:22:55.484201 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:22:55.485020 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:22:55.485023 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:22:55.485374 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:22:55.485375 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:22:55.485474 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:22:55.485964 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:22:55.485973 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:22:55.487335 (+ 1362us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:22:55.488890 (+ 1555us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:22:55.488897 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:22:55.488898 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:22:55.489127 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:22:55.489318 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:22:55.489319 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:22:55.489319 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:22:55.489422 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:22:55.489726 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:22:55.489732 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:22:55.489850 (+ 118us) client_negotiation.cc:769] Sending connection context
0812 01:22:55.489939 (+ 89us) client_negotiation.cc:240] Negotiation successful
0812 01:22:55.490058 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":114,"threads_started":1}
I20260812 01:22:55.490882 21457 heartbeater.cc:347] Connected to a master server at 127.24.0.190:45263
I20260812 01:22:55.490998 21457 heartbeater.cc:464] Registering TS with master...
I20260812 01:22:55.491031 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 21243
I20260812 01:22:55.491143 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786497534195564-24578-0/minicluster-data/ts-2/wal/instance
I20260812 01:22:55.491211 21457 heartbeater.cc:511] Master 127.24.0.190:45263 requested a full tablet report, sending...
I20260812 01:22:55.491339 24578 external_mini_cluster.cc:1428] Setting key 080d7ec472da3e94e880837bba9de1e8
I20260812 01:22:55.491638 20725 ts_manager.cc:194] Registered new tserver with Master: 34a95d3cc0a64b9d9c1e3a6eac3d4077 (127.24.0.131:41669)
I20260812 01:22:55.492201 20725 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.0.131@KRBTEST.COM'} at 127.24.0.131:47635
I20260812 01:22:55.492336 24578 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 01:22:55.499995 20792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:55.493885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47472 (local address 127.24.0.190:45263)
0812 01:22:55.493919 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0812 01:22:55.493921 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:22:55.494068 (+ 147us) server_negotiation.cc:407] Connection header received
0812 01:22:55.494150 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:22:55.494151 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:22:55.494161 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:22:55.494219 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:22:55.494758 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:55.495319 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:55.496100 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:55.496362 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:55.496436 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:22:55.497522 (+ 1086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:22:55.497524 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:22:55.497524 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:22:55.497531 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:22:55.498750 (+ 1219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:22:55.499164 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:22:55.499166 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:22:55.499166 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:22:55.499232 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:22:55.499439 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:22:55.499440 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:22:55.499441 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:22:55.499596 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:22:55.499710 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0812 01:22:55.499818 (+ 108us) server_negotiation.cc:299] Negotiation successful
0812 01:22:55.499896 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260812 01:22:55.501586 20725 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47472:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 01:22:55.503137 20725 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 01:22:55.509502 21479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:55.505437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.131:41669 (local address 127.0.0.1:38912)
0812 01:22:55.505812 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0812 01:22:55.505862 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 01:22:55.505973 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:22:55.506321 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:22:55.506322 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:22:55.506325 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:22:55.506387 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:22:55.506389 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:22:55.507285 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:22:55.507285 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:22:55.508421 (+ 1136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:22:55.508423 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:22:55.509207 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:22:55.509208 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:22:55.509291 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:22:55.509305 (+ 14us) client_negotiation.cc:769] Sending connection context
0812 01:22:55.509380 (+ 75us) client_negotiation.cc:240] Negotiation successful
0812 01:22:55.509428 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":340,"mutex_wait_us":5,"thread_start_us":84,"threads_started":1}
I20260812 01:22:55.509545 21481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:55.505530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38912 (local address 127.24.0.131:41669)
0812 01:22:55.505718 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0812 01:22:55.505723 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 01:22:55.505928 (+ 205us) server_negotiation.cc:407] Connection header received
0812 01:22:55.506120 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:22:55.506122 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:22:55.506140 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:22:55.506197 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:22:55.506474 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:55.507192 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:55.508504 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:55.509117 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:55.509195 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:22:55.509253 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 01:22:55.509411 (+ 158us) server_negotiation.cc:299] Negotiation successful
0812 01:22:55.509455 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":88,"threads_started":1}
I20260812 01:22:55.509958 21485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:55.505437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.129:39217 (local address 127.0.0.1:39642)
0812 01:22:55.505812 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0812 01:22:55.505859 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 01:22:55.505974 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:22:55.506298 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:22:55.506301 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:22:55.506304 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:22:55.506375 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:22:55.506376 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:22:55.507735 (+ 1359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:22:55.507736 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:22:55.508705 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:22:55.508707 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:22:55.509702 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:22:55.509703 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:22:55.509776 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:22:55.509779 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 01:22:55.509853 (+ 74us) client_negotiation.cc:240] Negotiation successful
0812 01:22:55.509893 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":72,"threads_started":1}
I20260812 01:22:55.509981 21482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:55.505574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39642 (local address 127.24.0.129:39217)
0812 01:22:55.505880 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0812 01:22:55.505884 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 01:22:55.505927 (+ 43us) server_negotiation.cc:407] Connection header received
0812 01:22:55.506108 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:22:55.506111 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:22:55.506129 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:22:55.506203 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:22:55.506467 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:55.507613 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:55.508785 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:55.509517 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:55.509591 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:22:55.509666 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 01:22:55.509871 (+ 205us) server_negotiation.cc:299] Negotiation successful
0812 01:22:55.509910 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":85,"threads_started":1}
I20260812 01:22:55.510636 21341 tablet_service.cc:1467] Processing CreateTablet for tablet 1ae5095eae13451a842bb2e388ea67be (DEFAULT_TABLE table=test-table [id=224796dff3f74a46b93bccf746376e25]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:22:55.510826 21480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:55.505661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.130:41905 (local address 127.0.0.1:50062)
0812 01:22:55.505927 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0812 01:22:55.505938 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 01:22:55.505979 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:22:55.506421 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:22:55.506422 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:22:55.506424 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:22:55.506466 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:22:55.506468 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:22:55.508776 (+ 2308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:22:55.508777 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:22:55.509587 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:22:55.509591 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:22:55.510596 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:22:55.510597 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:22:55.510659 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:22:55.510662 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 01:22:55.510721 (+ 59us) client_negotiation.cc:240] Negotiation successful
0812 01:22:55.510757 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":87,"threads_started":1}
I20260812 01:22:55.510885 21486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:55.505881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50062 (local address 127.24.0.130:41905)
0812 01:22:55.506203 (+ 322us) server_negotiation.cc:206] Beginning negotiation
0812 01:22:55.506208 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 01:22:55.506219 (+ 11us) server_negotiation.cc:407] Connection header received
0812 01:22:55.506256 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:22:55.506258 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:22:55.506274 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:22:55.506345 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:22:55.506538 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:55.507634 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:55.509692 (+ 2058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:55.510428 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:55.510492 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:22:55.510560 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 01:22:55.510757 (+ 197us) server_negotiation.cc:299] Negotiation successful
0812 01:22:55.510806 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":287,"thread_start_us":91,"threads_started":1}
I20260812 01:22:55.510901 20924 tablet_service.cc:1467] Processing CreateTablet for tablet 1ae5095eae13451a842bb2e388ea67be (DEFAULT_TABLE table=test-table [id=224796dff3f74a46b93bccf746376e25]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:22:55.511045 21341 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ae5095eae13451a842bb2e388ea67be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:22:55.511260 20924 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ae5095eae13451a842bb2e388ea67be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:22:55.511512 21121 tablet_service.cc:1467] Processing CreateTablet for tablet 1ae5095eae13451a842bb2e388ea67be (DEFAULT_TABLE table=test-table [id=224796dff3f74a46b93bccf746376e25]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:22:55.511827 21121 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ae5095eae13451a842bb2e388ea67be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:22:55.514006 21488 tablet_bootstrap.cc:492] T 1ae5095eae13451a842bb2e388ea67be P 12cb390782f546369937112c759df59e: Bootstrap starting.
I20260812 01:22:55.514657 21489 tablet_bootstrap.cc:492] T 1ae5095eae13451a842bb2e388ea67be P 34a95d3cc0a64b9d9c1e3a6eac3d4077: Bootstrap starting.
I20260812 01:22:55.514775 21488 tablet_bootstrap.cc:654] T 1ae5095eae13451a842bb2e388ea67be P 12cb390782f546369937112c759df59e: Neither blocks nor log segments found. Creating new log.
I20260812 01:22:55.515213 21490 tablet_bootstrap.cc:492] T 1ae5095eae13451a842bb2e388ea67be P b5a20898f61c45e8b71f8d883e190a94: Bootstrap starting.
I20260812 01:22:55.515457 21489 tablet_bootstrap.cc:654] T 1ae5095eae13451a842bb2e388ea67be P 34a95d3cc0a64b9d9c1e3a6eac3d4077: Neither blocks nor log segments found. Creating new log.
I20260812 01:22:55.515707 21488 log.cc:824] T 1ae5095eae13451a842bb2e388ea67be P 12cb390782f546369937112c759df59e: Log is configured to *not* fsync() on all Append() calls
I20260812 01:22:55.515945 21490 tablet_bootstrap.cc:654] T 1ae5095eae13451a842bb2e388ea67be P b5a20898f61c45e8b71f8d883e190a94: Neither blocks nor log segments found. Creating new log.
I20260812 01:22:55.516178 21489 log.cc:824] T 1ae5095eae13451a842bb2e388ea67be P 34a95d3cc0a64b9d9c1e3a6eac3d4077: Log is configured to *not* fsync() on all Append() calls
I20260812 01:22:55.516685 21488 tablet_bootstrap.cc:492] T 1ae5095eae13451a842bb2e388ea67be P 12cb390782f546369937112c759df59e: No bootstrap required, opened a new log
I20260812 01:22:55.516685 21490 log.cc:824] T 1ae5095eae13451a842bb2e388ea67be P b5a20898f61c45e8b71f8d883e190a94: Log is configured to *not* fsync() on all Append() calls
I20260812 01:22:55.516774 21488 ts_tablet_manager.cc:1397] T 1ae5095eae13451a842bb2e388ea67be P 12cb390782f546369937112c759df59e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 01:22:55.517046 21489 tablet_bootstrap.cc:492] T 1ae5095eae13451a842bb2e388ea67be P 34a95d3cc0a64b9d9c1e3a6eac3d4077: No bootstrap required, opened a new log
I20260812 01:22:55.517117 21489 ts_tablet_manager.cc:1397] T 1ae5095eae13451a842bb2e388ea67be P 34a95d3cc0a64b9d9c1e3a6eac3d4077: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260812 01:22:55.517700 21490 tablet_bootstrap.cc:492] T 1ae5095eae13451a842bb2e388ea67be P b5a20898f61c45e8b71f8d883e190a94: No bootstrap required, opened a new log
I20260812 01:22:55.517757 21490 ts_tablet_manager.cc:1397] T 1ae5095eae13451a842bb2e388ea67be P b5a20898f61c45e8b71f8d883e190a94: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 01:22:55.518321 21488 raft_consensus.cc:348] T 1ae5095eae13451a842bb2e388ea67be P 12cb390782f546369937112c759df59e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34a95d3cc0a64b9d9c1e3a6eac3d4077" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 41669 } } peers { permanent_uuid: "12cb390782f546369937112c759df59e" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 39217 } } peers { permanent_uuid: "b5a20898f61c45e8b71f8d883e190a94" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 41905 } }
I20260812 01:22:55.518446 21488 raft_consensus.cc:374] T 1ae5095eae13451a842bb2e388ea67be P 12cb390782f546369937112c759df59e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:22:55.518467 21488 raft_consensus.cc:729] T 1ae5095eae13451a842bb2e388ea67be P 12cb390782f546369937112c759df59e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12cb390782f546369937112c759df59e, State: Initialized, Role: FOLLOWER
I20260812 01:22:55.518595 21488 consensus_queue.cc:260] T 1ae5095eae13451a842bb2e388ea67be P 12cb390782f546369937112c759df59e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34a95d3cc0a64b9d9c1e3a6eac3d4077" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 41669 } } peers { permanent_uuid: "12cb390782f546369937112c759df59e" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 39217 } } peers { permanent_uuid: "b5a20898f61c45e8b71f8d883e190a94" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 41905 } }
I20260812 01:22:55.518882 21488 ts_tablet_manager.cc:1428] T 1ae5095eae13451a842bb2e388ea67be P 12cb390782f546369937112c759df59e: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 01:22:55.518998 21016 heartbeater.cc:503] Master 127.24.0.190:45263 was elected leader, sending a full tablet report...
I20260812 01:22:55.519160 21489 raft_consensus.cc:348] T 1ae5095eae13451a842bb2e388ea67be P 34a95d3cc0a64b9d9c1e3a6eac3d4077 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34a95d3cc0a64b9d9c1e3a6eac3d4077" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 41669 } } peers { permanent_uuid: "12cb390782f546369937112c759df59e" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 39217 } } peers { permanent_uuid: "b5a20898f61c45e8b71f8d883e190a94" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 41905 } }
I20260812 01:22:55.519320 21489 raft_consensus.cc:374] T 1ae5095eae13451a842bb2e388ea67be P 34a95d3cc0a64b9d9c1e3a6eac3d4077 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:22:55.519357 21489 raft_consensus.cc:729] T 1ae5095eae13451a842bb2e388ea67be P 34a95d3cc0a64b9d9c1e3a6eac3d4077 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34a95d3cc0a64b9d9c1e3a6eac3d4077, State: Initialized, Role: FOLLOWER
I20260812 01:22:55.519474 21489 consensus_queue.cc:260] T 1ae5095eae13451a842bb2e388ea67be P 34a95d3cc0a64b9d9c1e3a6eac3d4077 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34a95d3cc0a64b9d9c1e3a6eac3d4077" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 41669 } } peers { permanent_uuid: "12cb390782f546369937112c759df59e" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 39217 } } peers { permanent_uuid: "b5a20898f61c45e8b71f8d883e190a94" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 41905 } }
I20260812 01:22:55.519742 21489 ts_tablet_manager.cc:1428] T 1ae5095eae13451a842bb2e388ea67be P 34a95d3cc0a64b9d9c1e3a6eac3d4077: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260812 01:22:55.519747 21490 raft_consensus.cc:348] T 1ae5095eae13451a842bb2e388ea67be P b5a20898f61c45e8b71f8d883e190a94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34a95d3cc0a64b9d9c1e3a6eac3d4077" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 41669 } } peers { permanent_uuid: "12cb390782f546369937112c759df59e" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 39217 } } peers { permanent_uuid: "b5a20898f61c45e8b71f8d883e190a94" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 41905 } }
I20260812 01:22:55.519932 21457 heartbeater.cc:503] Master 127.24.0.190:45263 was elected leader, sending a full tablet report...
I20260812 01:22:55.519932 21490 raft_consensus.cc:374] T 1ae5095eae13451a842bb2e388ea67be P b5a20898f61c45e8b71f8d883e190a94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:22:55.520009 21490 raft_consensus.cc:729] T 1ae5095eae13451a842bb2e388ea67be P b5a20898f61c45e8b71f8d883e190a94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5a20898f61c45e8b71f8d883e190a94, State: Initialized, Role: FOLLOWER
I20260812 01:22:55.520200 21490 consensus_queue.cc:260] T 1ae5095eae13451a842bb2e388ea67be P b5a20898f61c45e8b71f8d883e190a94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34a95d3cc0a64b9d9c1e3a6eac3d4077" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 41669 } } peers { permanent_uuid: "12cb390782f546369937112c759df59e" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 39217 } } peers { permanent_uuid: "b5a20898f61c45e8b71f8d883e190a94" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 41905 } }
I20260812 01:22:55.520503 21490 ts_tablet_manager.cc:1428] T 1ae5095eae13451a842bb2e388ea67be P b5a20898f61c45e8b71f8d883e190a94: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 01:22:55.520514 21232 heartbeater.cc:503] Master 127.24.0.190:45263 was elected leader, sending a full tablet report...
W20260812 01:22:55.563580 21233 tablet.cc:2369] T 1ae5095eae13451a842bb2e388ea67be P b5a20898f61c45e8b71f8d883e190a94: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 01:22:55.663347 21018 tablet.cc:2369] T 1ae5095eae13451a842bb2e388ea67be P 12cb390782f546369937112c759df59e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 01:22:55.668455 21498 raft_consensus.cc:482] T 1ae5095eae13451a842bb2e388ea67be P 34a95d3cc0a64b9d9c1e3a6eac3d4077 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 01:22:55.668577 21498 raft_consensus.cc:504] T 1ae5095eae13451a842bb2e388ea67be P 34a95d3cc0a64b9d9c1e3a6eac3d4077 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34a95d3cc0a64b9d9c1e3a6eac3d4077" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 41669 } } peers { permanent_uuid: "12cb390782f546369937112c759df59e" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 39217 } } peers { permanent_uuid: "b5a20898f61c45e8b71f8d883e190a94" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 41905 } }
I20260812 01:22:55.669032 21498 leader_election.cc:288] T 1ae5095eae13451a842bb2e388ea67be P 34a95d3cc0a64b9d9c1e3a6eac3d4077 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12cb390782f546369937112c759df59e (127.24.0.129:39217), b5a20898f61c45e8b71f8d883e190a94 (127.24.0.130:41905)
I20260812 01:22:55.672498 21482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:55.669353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.131:53831 (local address 127.24.0.129:39217)
0812 01:22:55.669405 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 01:22:55.669408 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:22:55.669421 (+ 13us) server_negotiation.cc:407] Connection header received
0812 01:22:55.669519 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:22:55.669521 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:22:55.669538 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:22:55.669616 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:22:55.669821 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:55.670383 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:55.671654 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:55.672067 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:55.672175 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:22:55.672184 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 01:22:55.672381 (+ 197us) server_negotiation.cc:299] Negotiation successful
0812 01:22:55.672421 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260812 01:22:55.672499 21460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:55.669247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.129:39217 (local address 127.24.0.131:53831)
0812 01:22:55.669329 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0812 01:22:55.669347 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 01:22:55.669419 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:22:55.669635 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:22:55.669636 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:22:55.669638 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:22:55.669728 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:22:55.669730 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:22:55.670468 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:22:55.670469 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:22:55.671548 (+ 1079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:22:55.671551 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:22:55.672204 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:22:55.672205 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:22:55.672291 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:22:55.672294 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 01:22:55.672351 (+ 57us) client_negotiation.cc:240] Negotiation successful
0812 01:22:55.672412 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51,"mutex_wait_us":30}
I20260812 01:22:55.672511 21486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:55.669497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.131:60399 (local address 127.24.0.130:41905)
0812 01:22:55.669572 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0812 01:22:55.669574 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:22:55.669665 (+ 91us) server_negotiation.cc:407] Connection header received
0812 01:22:55.669721 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:22:55.669721 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:22:55.669734 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:22:55.669796 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:22:55.669965 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:55.670408 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:55.671614 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:55.672156 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:55.672225 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:22:55.672234 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 01:22:55.672410 (+ 176us) server_negotiation.cc:299] Negotiation successful
0812 01:22:55.672450 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 01:22:55.672516 21500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:55.669403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.130:41905 (local address 127.24.0.131:60399)
0812 01:22:55.669572 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0812 01:22:55.669583 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 01:22:55.669658 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:22:55.669820 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:22:55.669821 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:22:55.669823 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:22:55.669892 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:22:55.669893 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:22:55.670519 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:22:55.670520 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:22:55.671507 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:22:55.671509 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:22:55.672240 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:22:55.672241 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:22:55.672312 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:22:55.672316 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 01:22:55.672395 (+ 79us) client_negotiation.cc:240] Negotiation successful
0812 01:22:55.672429 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":99,"threads_started":1}
I20260812 01:22:55.672957 21148 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ae5095eae13451a842bb2e388ea67be" candidate_uuid: "34a95d3cc0a64b9d9c1e3a6eac3d4077" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5a20898f61c45e8b71f8d883e190a94" is_pre_election: true
I20260812 01:22:55.673007 20949 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ae5095eae13451a842bb2e388ea67be" candidate_uuid: "34a95d3cc0a64b9d9c1e3a6eac3d4077" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12cb390782f546369937112c759df59e" is_pre_election: true
I20260812 01:22:55.673152 21148 raft_consensus.cc:2454] T 1ae5095eae13451a842bb2e388ea67be P b5a20898f61c45e8b71f8d883e190a94 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a95d3cc0a64b9d9c1e3a6eac3d4077 in term 0.
I20260812 01:22:55.673151 20949 raft_consensus.cc:2454] T 1ae5095eae13451a842bb2e388ea67be P 12cb390782f546369937112c759df59e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a95d3cc0a64b9d9c1e3a6eac3d4077 in term 0.
I20260812 01:22:55.673383 21273 leader_election.cc:302] T 1ae5095eae13451a842bb2e388ea67be P 34a95d3cc0a64b9d9c1e3a6eac3d4077 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a95d3cc0a64b9d9c1e3a6eac3d4077, b5a20898f61c45e8b71f8d883e190a94; no voters:
I20260812 01:22:55.673509 21498 raft_consensus.cc:2781] T 1ae5095eae13451a842bb2e388ea67be P 34a95d3cc0a64b9d9c1e3a6eac3d4077 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 01:22:55.673568 21498 raft_consensus.cc:482] T 1ae5095eae13451a842bb2e388ea67be P 34a95d3cc0a64b9d9c1e3a6eac3d4077 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 01:22:55.673593 21498 raft_consensus.cc:3037] T 1ae5095eae13451a842bb2e388ea67be P 34a95d3cc0a64b9d9c1e3a6eac3d4077 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:22:55.674317 21498 raft_consensus.cc:504] T 1ae5095eae13451a842bb2e388ea67be P 34a95d3cc0a64b9d9c1e3a6eac3d4077 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34a95d3cc0a64b9d9c1e3a6eac3d4077" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 41669 } } peers { permanent_uuid: "12cb390782f546369937112c759df59e" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 39217 } } peers { permanent_uuid: "b5a20898f61c45e8b71f8d883e190a94" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 41905 } }
I20260812 01:22:55.674453 21498 leader_election.cc:288] T 1ae5095eae13451a842bb2e388ea67be P 34a95d3cc0a64b9d9c1e3a6eac3d4077 [CANDIDATE]: Term 1 election: Requested vote from peers 12cb390782f546369937112c759df59e (127.24.0.129:39217), b5a20898f61c45e8b71f8d883e190a94 (127.24.0.130:41905)
I20260812 01:22:55.674654 20949 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ae5095eae13451a842bb2e388ea67be" candidate_uuid: "34a95d3cc0a64b9d9c1e3a6eac3d4077" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12cb390782f546369937112c759df59e"
I20260812 01:22:55.674755 20949 raft_consensus.cc:3037] T 1ae5095eae13451a842bb2e388ea67be P 12cb390782f546369937112c759df59e [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:22:55.675459 20949 raft_consensus.cc:2454] T 1ae5095eae13451a842bb2e388ea67be P 12cb390782f546369937112c759df59e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a95d3cc0a64b9d9c1e3a6eac3d4077 in term 1.
I20260812 01:22:55.675595 21148 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ae5095eae13451a842bb2e388ea67be" candidate_uuid: "34a95d3cc0a64b9d9c1e3a6eac3d4077" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5a20898f61c45e8b71f8d883e190a94"
I20260812 01:22:55.675663 21273 leader_election.cc:302] T 1ae5095eae13451a842bb2e388ea67be P 34a95d3cc0a64b9d9c1e3a6eac3d4077 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12cb390782f546369937112c759df59e, 34a95d3cc0a64b9d9c1e3a6eac3d4077; no voters:
I20260812 01:22:55.675690 21148 raft_consensus.cc:3037] T 1ae5095eae13451a842bb2e388ea67be P b5a20898f61c45e8b71f8d883e190a94 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:22:55.675762 21498 raft_consensus.cc:2781] T 1ae5095eae13451a842bb2e388ea67be P 34a95d3cc0a64b9d9c1e3a6eac3d4077 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:22:55.675930 21498 raft_consensus.cc:686] T 1ae5095eae13451a842bb2e388ea67be P 34a95d3cc0a64b9d9c1e3a6eac3d4077 [term 1 LEADER]: Becoming Leader. State: Replica: 34a95d3cc0a64b9d9c1e3a6eac3d4077, State: Running, Role: LEADER
I20260812 01:22:55.676039 21498 consensus_queue.cc:237] T 1ae5095eae13451a842bb2e388ea67be P 34a95d3cc0a64b9d9c1e3a6eac3d4077 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34a95d3cc0a64b9d9c1e3a6eac3d4077" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 41669 } } peers { permanent_uuid: "12cb390782f546369937112c759df59e" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 39217 } } peers { permanent_uuid: "b5a20898f61c45e8b71f8d883e190a94" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 41905 } }
I20260812 01:22:55.676684 21148 raft_consensus.cc:2454] T 1ae5095eae13451a842bb2e388ea67be P b5a20898f61c45e8b71f8d883e190a94 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a95d3cc0a64b9d9c1e3a6eac3d4077 in term 1.
I20260812 01:22:55.677119 20722 catalog_manager.cc:5495] T 1ae5095eae13451a842bb2e388ea67be P 34a95d3cc0a64b9d9c1e3a6eac3d4077 reported cstate change: term changed from 0 to 1, leader changed from <none> to 34a95d3cc0a64b9d9c1e3a6eac3d4077 (127.24.0.131). New cstate: current_term: 1 leader_uuid: "34a95d3cc0a64b9d9c1e3a6eac3d4077" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34a95d3cc0a64b9d9c1e3a6eac3d4077" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 41669 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12cb390782f546369937112c759df59e" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 39217 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5a20898f61c45e8b71f8d883e190a94" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 41905 } health_report { overall_health: UNKNOWN } } }
I20260812 01:22:55.692600 21481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:55.690177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38924 (local address 127.24.0.131:41669)
0812 01:22:55.690240 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0812 01:22:55.690242 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:22:55.690251 (+ 9us) server_negotiation.cc:407] Connection header received
0812 01:22:55.690286 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:22:55.690287 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:22:55.690306 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:22:55.690389 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:22:55.690545 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:55.691014 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:55.691768 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:55.691979 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:55.692058 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:22:55.692247 (+ 189us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:22:55.692357 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:22:55.692454 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0812 01:22:55.692494 (+ 40us) server_negotiation.cc:299] Negotiation successful
0812 01:22:55.692520 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 01:22:55.694409 20949 raft_consensus.cc:1261] T 1ae5095eae13451a842bb2e388ea67be P 12cb390782f546369937112c759df59e [term 1 FOLLOWER]: Refusing update from remote peer 34a95d3cc0a64b9d9c1e3a6eac3d4077: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:22:55.694423 21148 raft_consensus.cc:1261] T 1ae5095eae13451a842bb2e388ea67be P b5a20898f61c45e8b71f8d883e190a94 [term 1 FOLLOWER]: Refusing update from remote peer 34a95d3cc0a64b9d9c1e3a6eac3d4077: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:22:55.694809 21501 consensus_queue.cc:1035] T 1ae5095eae13451a842bb2e388ea67be P 34a95d3cc0a64b9d9c1e3a6eac3d4077 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5a20898f61c45e8b71f8d883e190a94" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 41905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:22:55.694942 21498 consensus_queue.cc:1035] T 1ae5095eae13451a842bb2e388ea67be P 34a95d3cc0a64b9d9c1e3a6eac3d4077 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12cb390782f546369937112c759df59e" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 39217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:22:55.697760 21508 mvcc.cc:205] Tried to move back new op lower bound from 7317494889241595904 to 7317494889170026496. Current Snapshot: MvccSnapshot[applied={T|T < 7317494889241595904}]
I20260812 01:22:55.698096 21506 mvcc.cc:205] Tried to move back new op lower bound from 7317494889241595904 to 7317494889170026496. Current Snapshot: MvccSnapshot[applied={T|T < 7317494889241595904}]
I20260812 01:22:55.699533 20722 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47472:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:22:55.699636 20722 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47472:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:22:55.700420 20722 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9842a47e153a4d138ec8dd2baf1b2cb3: Sending DeleteTablet for 3 replicas of tablet 1ae5095eae13451a842bb2e388ea67be
I20260812 01:22:55.700744 21341 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ae5095eae13451a842bb2e388ea67be with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:22:55 UTC) from {username='kudu', principal='kudu/127.24.0.190@KRBTEST.COM'} at 127.0.0.1:38912
I20260812 01:22:55.700786 21121 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ae5095eae13451a842bb2e388ea67be with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:22:55 UTC) from {username='kudu', principal='kudu/127.24.0.190@KRBTEST.COM'} at 127.0.0.1:50062
I20260812 01:22:55.700798 20924 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ae5095eae13451a842bb2e388ea67be with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:22:55 UTC) from {username='kudu', principal='kudu/127.24.0.190@KRBTEST.COM'} at 127.0.0.1:39642
I20260812 01:22:55.701050 21522 tablet_replica.cc:330] T 1ae5095eae13451a842bb2e388ea67be P 34a95d3cc0a64b9d9c1e3a6eac3d4077: stopping tablet replica
I20260812 01:22:55.701156 21522 raft_consensus.cc:2229] T 1ae5095eae13451a842bb2e388ea67be P 34a95d3cc0a64b9d9c1e3a6eac3d4077 [term 1 LEADER]: Raft consensus shutting down.
I20260812 01:22:55.701265 21522 raft_consensus.cc:2258] T 1ae5095eae13451a842bb2e388ea67be P 34a95d3cc0a64b9d9c1e3a6eac3d4077 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:22:55.701303 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 20829
I20260812 01:22:55.710106 21524 tablet_replica.cc:330] T 1ae5095eae13451a842bb2e388ea67be P b5a20898f61c45e8b71f8d883e190a94: stopping tablet replica
I20260812 01:22:55.710217 21524 raft_consensus.cc:2229] T 1ae5095eae13451a842bb2e388ea67be P b5a20898f61c45e8b71f8d883e190a94 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 01:22:55.710310 21524 raft_consensus.cc:2258] T 1ae5095eae13451a842bb2e388ea67be P b5a20898f61c45e8b71f8d883e190a94 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:22:55.711336 21522 ts_tablet_manager.cc:1905] T 1ae5095eae13451a842bb2e388ea67be P 34a95d3cc0a64b9d9c1e3a6eac3d4077: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:22:55.711706 21524 ts_tablet_manager.cc:1905] T 1ae5095eae13451a842bb2e388ea67be P b5a20898f61c45e8b71f8d883e190a94: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:22:55.713259 21522 ts_tablet_manager.cc:1918] T 1ae5095eae13451a842bb2e388ea67be P 34a95d3cc0a64b9d9c1e3a6eac3d4077: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 01:22:55.713413 21522 log.cc:1196] T 1ae5095eae13451a842bb2e388ea67be P 34a95d3cc0a64b9d9c1e3a6eac3d4077: Deleting WAL directory at /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786497534195564-24578-0/minicluster-data/ts-2/wal/wals/1ae5095eae13451a842bb2e388ea67be
I20260812 01:22:55.713449 21524 ts_tablet_manager.cc:1918] T 1ae5095eae13451a842bb2e388ea67be P b5a20898f61c45e8b71f8d883e190a94: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 01:22:55.713626 21524 log.cc:1196] T 1ae5095eae13451a842bb2e388ea67be P b5a20898f61c45e8b71f8d883e190a94: Deleting WAL directory at /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786497534195564-24578-0/minicluster-data/ts-1/wal/wals/1ae5095eae13451a842bb2e388ea67be
I20260812 01:22:55.713778 21522 ts_tablet_manager.cc:1939] T 1ae5095eae13451a842bb2e388ea67be P 34a95d3cc0a64b9d9c1e3a6eac3d4077: Deleting consensus metadata
I20260812 01:22:55.713990 21524 ts_tablet_manager.cc:1939] T 1ae5095eae13451a842bb2e388ea67be P b5a20898f61c45e8b71f8d883e190a94: Deleting consensus metadata
I20260812 01:22:55.714283 20698 catalog_manager.cc:4841] TS 34a95d3cc0a64b9d9c1e3a6eac3d4077 (127.24.0.131:41669): tablet 1ae5095eae13451a842bb2e388ea67be (table test-table [id=224796dff3f74a46b93bccf746376e25]) successfully deleted
I20260812 01:22:55.714800 20703 catalog_manager.cc:4841] TS b5a20898f61c45e8b71f8d883e190a94 (127.24.0.130:41905): tablet 1ae5095eae13451a842bb2e388ea67be (table test-table [id=224796dff3f74a46b93bccf746376e25]) successfully deleted
W20260812 01:22:55.717025 20703 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.0.129:39217 (error 108)
I20260812 01:22:55.717630 21480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:55.717470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.129:39217 (local address 127.0.0.1:39656)
0812 01:22:55.717540 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0812 01:22:55.717587 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.129:39217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260812 01:22:55.717753 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 21022
W20260812 01:22:55.717795 20703 catalog_manager.cc:4584] TS 12cb390782f546369937112c759df59e (127.24.0.129:39217): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1ae5095eae13451a842bb2e388ea67be: Network error: Client connection negotiation failed: client connection to 127.24.0.129:39217: connect: Connection refused (error 111)
I20260812 01:22:55.739598 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 21243
I20260812 01:22:55.751170 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 20673
2026-08-12T01:22:55Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2345 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-12T01:22:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:22:55Z Disabled control of system clock
I20260812 01:22:55.788197 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786497534195564-24578-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786497534195564-24578-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786497534195564-24578-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.190:44097
--webserver_interface=127.24.0.190
--webserver_port=0
--builtin_ntp_servers=127.24.0.148:38709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.0.190:44097
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 01:22:55.870014 21535 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:22:55.870229 21535 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:22:55.870250 21535 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:22:55.871304 21535 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:22:55.871336 21535 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:22:55.871349 21535 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:22:55.871362 21535 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:22:55.871374 21535 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:22:55.872726 21535 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:38709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786497534195564-24578-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786497534195564-24578-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.0.190:44097
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.190:44097
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.0.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21535
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786497534195564-24578-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:22:55.872969 21535 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:22:55.873232 21535 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:22:55.878129 21545 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:22:55.878137 21542 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:22:55.878257 21535 server_base.cc:1034] running on GCE node
W20260812 01:22:55.878137 21547 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:22:55.878779 21535 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:22:55.879083 21535 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:22:55.880228 21535 hybrid_clock.cc:648] HybridClock initialized: now 1786497775880216 us; error 32 us; skew 500 ppm
I20260812 01:22:55.881554 21535 webserver.cc:460] Webserver started at http://127.24.0.190:38071/ using document root <none> and password file <none>
I20260812 01:22:55.881832 21535 fs_manager.cc:359] Metadata directory not provided
I20260812 01:22:55.881908 21535 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:22:55.882066 21535 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:22:55.883189 21535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786497534195564-24578-0/minicluster-data/master-0/data/instance:
uuid: "fe886efefe78449ca8b658168cb324a1"
format_stamp: "Formatted at 2026-08-12 01:22:55 on dist-test-slave-t3q3"
server_key: "530b11494045630bf320b9b875c08cc7"
server_key_iv: "d11649c80df2b1ad520b1756bbeaf49e"
server_key_version: "encryptionkey@0"
I20260812 01:22:55.883569 21535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786497534195564-24578-0/minicluster-data/master-0/wal/instance:
uuid: "fe886efefe78449ca8b658168cb324a1"
format_stamp: "Formatted at 2026-08-12 01:22:55 on dist-test-slave-t3q3"
server_key: "530b11494045630bf320b9b875c08cc7"
server_key_iv: "d11649c80df2b1ad520b1756bbeaf49e"
server_key_version: "encryptionkey@0"
I20260812 01:22:55.884986 21535 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 01:22:55.885776 21553 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:22:55.885963 21535 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 01:22:55.886073 21535 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786497534195564-24578-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786497534195564-24578-0/minicluster-data/master-0/wal
uuid: "fe886efefe78449ca8b658168cb324a1"
format_stamp: "Formatted at 2026-08-12 01:22:55 on dist-test-slave-t3q3"
server_key: "530b11494045630bf320b9b875c08cc7"
server_key_iv: "d11649c80df2b1ad520b1756bbeaf49e"
server_key_version: "encryptionkey@0"
I20260812 01:22:55.886152 21535 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786497534195564-24578-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786497534195564-24578-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786497534195564-24578-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:22:55.893543 21535 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:22:55.893841 21535 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:22:55.893996 21535 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:22:55.898162 21535 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.190:44097
I20260812 01:22:55.898192 21642 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.190:44097 every 8 connection(s)
I20260812 01:22:55.898535 21535 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
I20260812 01:22:55.899518 21643 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:22:55.901870 21643 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe886efefe78449ca8b658168cb324a1: Bootstrap starting.
I20260812 01:22:55.902484 21643 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fe886efefe78449ca8b658168cb324a1: Neither blocks nor log segments found. Creating new log.
I20260812 01:22:55.902755 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 21535
I20260812 01:22:55.902873 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786497534195564-24578-0/minicluster-data/master-0/wal/instance
I20260812 01:22:55.902885 21643 log.cc:824] T 00000000000000000000000000000000 P fe886efefe78449ca8b658168cb324a1: Log is configured to *not* fsync() on all Append() calls
I20260812 01:22:55.903090 24578 external_mini_cluster.cc:1428] Setting key 79213b636a6f4921d90a93925feaa6ed
I20260812 01:22:55.903735 21643 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe886efefe78449ca8b658168cb324a1: No bootstrap required, opened a new log
I20260812 01:22:55.906613 21643 raft_consensus.cc:348] T 00000000000000000000000000000000 P fe886efefe78449ca8b658168cb324a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe886efefe78449ca8b658168cb324a1" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 44097 } }
I20260812 01:22:55.906778 21643 raft_consensus.cc:374] T 00000000000000000000000000000000 P fe886efefe78449ca8b658168cb324a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:22:55.906807 21643 raft_consensus.cc:729] T 00000000000000000000000000000000 P fe886efefe78449ca8b658168cb324a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe886efefe78449ca8b658168cb324a1, State: Initialized, Role: FOLLOWER
I20260812 01:22:55.906919 21643 consensus_queue.cc:260] T 00000000000000000000000000000000 P fe886efefe78449ca8b658168cb324a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe886efefe78449ca8b658168cb324a1" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 44097 } }
I20260812 01:22:55.907006 21643 raft_consensus.cc:388] T 00000000000000000000000000000000 P fe886efefe78449ca8b658168cb324a1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:22:55.907099 21643 raft_consensus.cc:482] T 00000000000000000000000000000000 P fe886efefe78449ca8b658168cb324a1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:22:55.907182 21643 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fe886efefe78449ca8b658168cb324a1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:22:55.907720 21643 raft_consensus.cc:504] T 00000000000000000000000000000000 P fe886efefe78449ca8b658168cb324a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe886efefe78449ca8b658168cb324a1" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 44097 } }
I20260812 01:22:55.907846 21643 leader_election.cc:302] T 00000000000000000000000000000000 P fe886efefe78449ca8b658168cb324a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fe886efefe78449ca8b658168cb324a1; no voters:
I20260812 01:22:55.908035 21643 leader_election.cc:288] T 00000000000000000000000000000000 P fe886efefe78449ca8b658168cb324a1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 01:22:55.908107 21650 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fe886efefe78449ca8b658168cb324a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:22:55.908231 21650 raft_consensus.cc:686] T 00000000000000000000000000000000 P fe886efefe78449ca8b658168cb324a1 [term 1 LEADER]: Becoming Leader. State: Replica: fe886efefe78449ca8b658168cb324a1, State: Running, Role: LEADER
I20260812 01:22:55.908329 21643 sys_catalog.cc:564] T 00000000000000000000000000000000 P fe886efefe78449ca8b658168cb324a1 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:22:55.908406 21650 consensus_queue.cc:237] T 00000000000000000000000000000000 P fe886efefe78449ca8b658168cb324a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe886efefe78449ca8b658168cb324a1" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 44097 } }
I20260812 01:22:55.908746 21650 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe886efefe78449ca8b658168cb324a1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fe886efefe78449ca8b658168cb324a1. Latest consensus state: current_term: 1 leader_uuid: "fe886efefe78449ca8b658168cb324a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe886efefe78449ca8b658168cb324a1" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 44097 } } }
I20260812 01:22:55.908885 21650 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe886efefe78449ca8b658168cb324a1 [sys.catalog]: This master's current role is: LEADER
I20260812 01:22:55.908946 21655 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe886efefe78449ca8b658168cb324a1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fe886efefe78449ca8b658168cb324a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe886efefe78449ca8b658168cb324a1" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 44097 } } }
I20260812 01:22:55.909040 21655 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe886efefe78449ca8b658168cb324a1 [sys.catalog]: This master's current role is: LEADER
I20260812 01:22:55.909726 21666 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:22:55.910449 21649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:55.903691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33350 (local address 127.24.0.190:44097)
0812 01:22:55.903970 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0812 01:22:55.903978 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 01:22:55.904022 (+ 44us) server_negotiation.cc:407] Connection header received
0812 01:22:55.905041 (+ 1019us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:22:55.905051 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:22:55.905126 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:22:55.905274 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:22:55.905467 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:55.906933 (+ 1466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:55.907855 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:55.908543 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:55.908664 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:22:55.909671 (+ 1007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:22:55.909675 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:22:55.909676 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:22:55.909885 (+ 209us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:22:55.909927 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:22:55.909943 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:22:55.909990 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0812 01:22:55.910157 (+ 167us) server_negotiation.cc:299] Negotiation successful
0812 01:22:55.910248 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":198,"thread_start_us":100,"threads_started":1}
I20260812 01:22:55.910473 21666 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:22:55.912350 21666 catalog_manager.cc:1347] Generated new cluster ID: 931990d93ed54063bafaf642dbc628aa
I20260812 01:22:55.912405 21666 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 01:22:55.922713 21666 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:22:55.923405 21666 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:22:55.941447 21666 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fe886efefe78449ca8b658168cb324a1: Generated new TSK 0
I20260812 01:22:55.941689 21666 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 01:22:55.942858 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786497534195564-24578-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786497534195564-24578-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786497534195564-24578-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786497534195564-24578-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.0.129:0
--local_ip_for_outbound_sockets=127.24.0.129
--webserver_interface=127.24.0.129
--webserver_port=0
--tserver_master_addrs=127.24.0.190:44097
--builtin_ntp_servers=127.24.0.148:38709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 01:22:56.027251 21686 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:22:56.027480 21686 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:22:56.027518 21686 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:22:56.028723 21686 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:22:56.028788 21686 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:22:56.028832 21686 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.0.129
I20260812 01:22:56.030208 21686 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:38709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786497534195564-24578-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786497534195564-24578-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.0.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786497534195564-24578-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.0.129
--webserver_port=0
--tserver_master_addrs=127.24.0.190:44097
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21686
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.0.129
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786497534195564-24578-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:22:56.030478 21686 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:22:56.030787 21686 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:22:56.035866 21696 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:22:56.035929 21698 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:22:56.035993 21686 server_base.cc:1034] running on GCE node
W20260812 01:22:56.036181 21700 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:22:56.036366 21686 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:22:56.036651 21686 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:22:56.037824 21686 hybrid_clock.cc:648] HybridClock initialized: now 1786497776037806 us; error 37 us; skew 500 ppm
I20260812 01:22:56.039160 21686 webserver.cc:460] Webserver started at http://127.24.0.129:35419/ using document root <none> and password file <none>
I20260812 01:22:56.039469 21686 fs_manager.cc:359] Metadata directory not provided
I20260812 01:22:56.039521 21686 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:22:56.039671 21686 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:22:56.040854 21686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786497534195564-24578-0/minicluster-data/ts-0/data/instance:
uuid: "69ca216d655348c29f93945a75dd4783"
format_stamp: "Formatted at 2026-08-12 01:22:56 on dist-test-slave-t3q3"
server_key: "05f8d2f2f2bd04c7a1bfcce770719dbc"
server_key_iv: "6ed50622728b423b20ac0be409733d55"
server_key_version: "encryptionkey@0"
I20260812 01:22:56.041225 21686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786497534195564-24578-0/minicluster-data/ts-0/wal/instance:
uuid: "69ca216d655348c29f93945a75dd4783"
format_stamp: "Formatted at 2026-08-12 01:22:56 on dist-test-slave-t3q3"
server_key: "05f8d2f2f2bd04c7a1bfcce770719dbc"
server_key_iv: "6ed50622728b423b20ac0be409733d55"
server_key_version: "encryptionkey@0"
I20260812 01:22:56.042625 21686 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 01:22:56.043406 21708 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:22:56.043604 21686 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:22:56.043694 21686 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786497534195564-24578-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786497534195564-24578-0/minicluster-data/ts-0/wal
uuid: "69ca216d655348c29f93945a75dd4783"
format_stamp: "Formatted at 2026-08-12 01:22:56 on dist-test-slave-t3q3"
server_key: "05f8d2f2f2bd04c7a1bfcce770719dbc"
server_key_iv: "6ed50622728b423b20ac0be409733d55"
server_key_version: "encryptionkey@0"
I20260812 01:22:56.043749 21686 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786497534195564-24578-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786497534195564-24578-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786497534195564-24578-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:22:56.061260 21686 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:22:56.061534 21686 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:22:56.061641 21686 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:22:56.061832 21686 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:22:56.062240 21686 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:22:56.062279 21686 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:22:56.062304 21686 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:22:56.062351 21686 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:22:56.068646 21686 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.129:34497
I20260812 01:22:56.068758 21892 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.129:34497 every 8 connection(s)
I20260812 01:22:56.069144 21686 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786497534195564-24578-0/minicluster-data/ts-0/data/info.pb
I20260812 01:22:56.074575 21649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:56.069935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.129:52411 (local address 127.24.0.190:44097)
0812 01:22:56.069988 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0812 01:22:56.069990 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:22:56.070214 (+ 224us) server_negotiation.cc:407] Connection header received
0812 01:22:56.070763 (+ 549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:22:56.070765 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:22:56.070785 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:22:56.070842 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:22:56.071585 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:56.072121 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:56.073142 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:56.073347 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:56.073430 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:22:56.074268 (+ 838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:22:56.074270 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:22:56.074270 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:22:56.074276 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:22:56.074291 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:22:56.074295 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:22:56.074352 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 01:22:56.074477 (+ 125us) server_negotiation.cc:299] Negotiation successful
0812 01:22:56.074504 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 01:22:56.074822 21895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:56.069896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:44097 (local address 127.24.0.129:52411)
0812 01:22:56.070080 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0812 01:22:56.070116 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 01:22:56.070646 (+ 530us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:22:56.070914 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:22:56.070918 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:22:56.070928 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:22:56.071463 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:22:56.071468 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:22:56.072231 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:22:56.072232 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:22:56.073043 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:22:56.073045 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:22:56.073439 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:22:56.073440 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:22:56.073583 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:22:56.074124 (+ 541us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 01:22:56.074124 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:22:56.074153 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 01:22:56.074176 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:22:56.074377 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:22:56.074384 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:22:56.074387 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 01:22:56.074461 (+ 74us) client_negotiation.cc:240] Negotiation successful
0812 01:22:56.074565 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":71,"threads_started":1}
I20260812 01:22:56.075356 21893 heartbeater.cc:347] Connected to a master server at 127.24.0.190:44097
I20260812 01:22:56.075451 21893 heartbeater.cc:464] Registering TS with master...
I20260812 01:22:56.075670 21893 heartbeater.cc:511] Master 127.24.0.190:44097 requested a full tablet report, sending...
I20260812 01:22:56.076313 21582 ts_manager.cc:194] Registered new tserver with Master: 69ca216d655348c29f93945a75dd4783 (127.24.0.129:34497)
I20260812 01:22:56.076882 21582 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.0.129:52411
I20260812 01:22:56.079282 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 21686
I20260812 01:22:56.079370 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786497534195564-24578-0/minicluster-data/ts-0/wal/instance
I20260812 01:22:56.079523 24578 external_mini_cluster.cc:1428] Setting key 2fd2f8d8d8972eed8b95e6cd5a5bb796
I20260812 01:22:56.080528 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786497534195564-24578-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786497534195564-24578-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786497534195564-24578-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786497534195564-24578-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.0.130:0
--local_ip_for_outbound_sockets=127.24.0.130
--webserver_interface=127.24.0.130
--webserver_port=0
--tserver_master_addrs=127.24.0.190:44097
--builtin_ntp_servers=127.24.0.148:38709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 01:22:56.165467 21896 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:22:56.165720 21896 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:22:56.165757 21896 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:22:56.166857 21896 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:22:56.166923 21896 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:22:56.166963 21896 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.0.130
I20260812 01:22:56.168334 21896 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:38709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786497534195564-24578-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786497534195564-24578-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.0.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786497534195564-24578-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.0.130
--webserver_port=0
--tserver_master_addrs=127.24.0.190:44097
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21896
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.0.130
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786497534195564-24578-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:22:56.168605 21896 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:22:56.168902 21896 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:22:56.174011 21906 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:22:56.173998 21902 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:22:56.174186 21896 server_base.cc:1034] running on GCE node
W20260812 01:22:56.174237 21910 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:22:56.174482 21896 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:22:56.174746 21896 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:22:56.175907 21896 hybrid_clock.cc:648] HybridClock initialized: now 1786497776175899 us; error 38 us; skew 500 ppm
I20260812 01:22:56.177325 21896 webserver.cc:460] Webserver started at http://127.24.0.130:39991/ using document root <none> and password file <none>
I20260812 01:22:56.177583 21896 fs_manager.cc:359] Metadata directory not provided
I20260812 01:22:56.177659 21896 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:22:56.177815 21896 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:22:56.178889 21896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786497534195564-24578-0/minicluster-data/ts-1/data/instance:
uuid: "ae7b2fcbdede4e71a5546877f6850599"
format_stamp: "Formatted at 2026-08-12 01:22:56 on dist-test-slave-t3q3"
server_key: "a6fc859604a9f3aa9508083c9d795214"
server_key_iv: "57db6606386bfd173e892fa3021f87d6"
server_key_version: "encryptionkey@0"
I20260812 01:22:56.179268 21896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786497534195564-24578-0/minicluster-data/ts-1/wal/instance:
uuid: "ae7b2fcbdede4e71a5546877f6850599"
format_stamp: "Formatted at 2026-08-12 01:22:56 on dist-test-slave-t3q3"
server_key: "a6fc859604a9f3aa9508083c9d795214"
server_key_iv: "57db6606386bfd173e892fa3021f87d6"
server_key_version: "encryptionkey@0"
I20260812 01:22:56.180708 21896 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260812 01:22:56.181452 21921 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:22:56.181633 21896 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 01:22:56.181747 21896 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786497534195564-24578-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786497534195564-24578-0/minicluster-data/ts-1/wal
uuid: "ae7b2fcbdede4e71a5546877f6850599"
format_stamp: "Formatted at 2026-08-12 01:22:56 on dist-test-slave-t3q3"
server_key: "a6fc859604a9f3aa9508083c9d795214"
server_key_iv: "57db6606386bfd173e892fa3021f87d6"
server_key_version: "encryptionkey@0"
I20260812 01:22:56.181823 21896 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786497534195564-24578-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786497534195564-24578-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786497534195564-24578-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:22:56.190990 21896 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:22:56.191291 21896 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:22:56.191437 21896 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:22:56.191653 21896 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:22:56.191977 21896 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:22:56.192041 21896 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:22:56.192123 21896 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:22:56.192149 21896 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:22:56.198175 21896 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.130:38947
I20260812 01:22:56.198257 22110 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.130:38947 every 8 connection(s)
I20260812 01:22:56.198601 21896 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786497534195564-24578-0/minicluster-data/ts-1/data/info.pb
I20260812 01:22:56.203962 21649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:56.199405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.130:45897 (local address 127.24.0.190:44097)
0812 01:22:56.199485 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0812 01:22:56.199488 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:22:56.199681 (+ 193us) server_negotiation.cc:407] Connection header received
0812 01:22:56.200303 (+ 622us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:22:56.200304 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:22:56.200318 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:22:56.200371 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:22:56.201064 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:56.201577 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:56.202580 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:56.202796 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:56.202876 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:22:56.203686 (+ 810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:22:56.203687 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:22:56.203688 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:22:56.203693 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:22:56.203708 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:22:56.203713 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:22:56.203762 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0812 01:22:56.203857 (+ 95us) server_negotiation.cc:299] Negotiation successful
0812 01:22:56.203883 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 01:22:56.204211 22115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:56.199305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:44097 (local address 127.24.0.130:45897)
0812 01:22:56.199552 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0812 01:22:56.199595 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 01:22:56.200177 (+ 582us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:22:56.200442 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:22:56.200446 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:22:56.200455 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:22:56.200961 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:22:56.200965 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:22:56.201681 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:22:56.201682 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:22:56.202462 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:22:56.202464 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:22:56.202884 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:22:56.202885 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:22:56.203030 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:22:56.203542 (+ 512us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 01:22:56.203542 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:22:56.203571 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 01:22:56.203593 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:22:56.203763 (+ 170us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:22:56.203768 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:22:56.203771 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 01:22:56.203858 (+ 87us) client_negotiation.cc:240] Negotiation successful
0812 01:22:56.203957 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":115,"threads_started":1}
I20260812 01:22:56.204736 22111 heartbeater.cc:347] Connected to a master server at 127.24.0.190:44097
I20260812 01:22:56.204814 22111 heartbeater.cc:464] Registering TS with master...
I20260812 01:22:56.204950 22111 heartbeater.cc:511] Master 127.24.0.190:44097 requested a full tablet report, sending...
I20260812 01:22:56.205319 21582 ts_manager.cc:194] Registered new tserver with Master: ae7b2fcbdede4e71a5546877f6850599 (127.24.0.130:38947)
I20260812 01:22:56.205713 21582 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.0.130:45897
I20260812 01:22:56.206418 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 21896
I20260812 01:22:56.206521 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786497534195564-24578-0/minicluster-data/ts-1/wal/instance
I20260812 01:22:56.206764 24578 external_mini_cluster.cc:1428] Setting key 8cd6afbc2e83d980bf222216b753783e
I20260812 01:22:56.207868 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786497534195564-24578-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786497534195564-24578-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786497534195564-24578-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786497534195564-24578-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.0.131:0
--local_ip_for_outbound_sockets=127.24.0.131
--webserver_interface=127.24.0.131
--webserver_port=0
--tserver_master_addrs=127.24.0.190:44097
--builtin_ntp_servers=127.24.0.148:38709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 01:22:56.291712 22116 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:22:56.291919 22116 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:22:56.291941 22116 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:22:56.293180 22116 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:22:56.293221 22116 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:22:56.293246 22116 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.0.131
I20260812 01:22:56.294605 22116 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:38709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786497534195564-24578-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786497534195564-24578-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.0.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786497534195564-24578-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.0.131
--webserver_port=0
--tserver_master_addrs=127.24.0.190:44097
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22116
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.0.131
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786497534195564-24578-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:22:56.294857 22116 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:22:56.295156 22116 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:22:56.300055 22131 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:22:56.300115 22122 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:22:56.300251 22116 server_base.cc:1034] running on GCE node
W20260812 01:22:56.300114 22123 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:22:56.300514 22116 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:22:56.300805 22116 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:22:56.301956 22116 hybrid_clock.cc:648] HybridClock initialized: now 1786497776301924 us; error 51 us; skew 500 ppm
I20260812 01:22:56.303248 22116 webserver.cc:460] Webserver started at http://127.24.0.131:34895/ using document root <none> and password file <none>
I20260812 01:22:56.303495 22116 fs_manager.cc:359] Metadata directory not provided
I20260812 01:22:56.303545 22116 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:22:56.303679 22116 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:22:56.304745 22116 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786497534195564-24578-0/minicluster-data/ts-2/data/instance:
uuid: "96427713b1344b6189e61adce4036328"
format_stamp: "Formatted at 2026-08-12 01:22:56 on dist-test-slave-t3q3"
server_key: "291968b3f8fbc8764b42cd3792effb81"
server_key_iv: "76930dae5f5b514cc4ba7cf7b72ba899"
server_key_version: "encryptionkey@0"
I20260812 01:22:56.305107 22116 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786497534195564-24578-0/minicluster-data/ts-2/wal/instance:
uuid: "96427713b1344b6189e61adce4036328"
format_stamp: "Formatted at 2026-08-12 01:22:56 on dist-test-slave-t3q3"
server_key: "291968b3f8fbc8764b42cd3792effb81"
server_key_iv: "76930dae5f5b514cc4ba7cf7b72ba899"
server_key_version: "encryptionkey@0"
I20260812 01:22:56.306404 22116 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 01:22:56.307158 22140 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:22:56.307339 22116 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 01:22:56.307428 22116 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786497534195564-24578-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786497534195564-24578-0/minicluster-data/ts-2/wal
uuid: "96427713b1344b6189e61adce4036328"
format_stamp: "Formatted at 2026-08-12 01:22:56 on dist-test-slave-t3q3"
server_key: "291968b3f8fbc8764b42cd3792effb81"
server_key_iv: "76930dae5f5b514cc4ba7cf7b72ba899"
server_key_version: "encryptionkey@0"
I20260812 01:22:56.307497 22116 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786497534195564-24578-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786497534195564-24578-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786497534195564-24578-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:22:56.318976 22116 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:22:56.319212 22116 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:22:56.319309 22116 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:22:56.319481 22116 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:22:56.319818 22116 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:22:56.319852 22116 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:22:56.319872 22116 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:22:56.319921 22116 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:22:56.325933 22116 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.131:34023
I20260812 01:22:56.326004 22328 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.131:34023 every 8 connection(s)
I20260812 01:22:56.326340 22116 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786497534195564-24578-0/minicluster-data/ts-2/data/info.pb
I20260812 01:22:56.331442 21649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:56.327080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.131:36489 (local address 127.24.0.190:44097)
0812 01:22:56.327158 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0812 01:22:56.327160 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:22:56.327339 (+ 179us) server_negotiation.cc:407] Connection header received
0812 01:22:56.327831 (+ 492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:22:56.327832 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:22:56.327843 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:22:56.327906 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:22:56.328587 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:56.329073 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:56.330051 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:56.330246 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:56.330301 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:22:56.331194 (+ 893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:22:56.331196 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:22:56.331196 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:22:56.331201 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:22:56.331217 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:22:56.331221 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:22:56.331276 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 01:22:56.331360 (+ 84us) server_negotiation.cc:299] Negotiation successful
0812 01:22:56.331384 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 01:22:56.331640 22332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:56.327023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:44097 (local address 127.24.0.131:36489)
0812 01:22:56.327221 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0812 01:22:56.327253 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0812 01:22:56.327736 (+ 483us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:22:56.327957 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:22:56.327960 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:22:56.327968 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:22:56.328495 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:22:56.328499 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:22:56.329159 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:22:56.329159 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:22:56.329951 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:22:56.329954 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:22:56.330333 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:22:56.330334 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:22:56.330446 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:22:56.331046 (+ 600us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 01:22:56.331047 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:22:56.331075 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 01:22:56.331098 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:22:56.331289 (+ 191us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:22:56.331294 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:22:56.331296 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 01:22:56.331349 (+ 53us) client_negotiation.cc:240] Negotiation successful
0812 01:22:56.331424 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":100,"threads_started":1}
I20260812 01:22:56.332278 22329 heartbeater.cc:347] Connected to a master server at 127.24.0.190:44097
I20260812 01:22:56.332386 22329 heartbeater.cc:464] Registering TS with master...
I20260812 01:22:56.332582 22329 heartbeater.cc:511] Master 127.24.0.190:44097 requested a full tablet report, sending...
I20260812 01:22:56.333048 21582 ts_manager.cc:194] Registered new tserver with Master: 96427713b1344b6189e61adce4036328 (127.24.0.131:34023)
I20260812 01:22:56.333420 21582 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.0.131:36489
I20260812 01:22:56.333781 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 22116
I20260812 01:22:56.333873 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786497534195564-24578-0/minicluster-data/ts-2/wal/instance
I20260812 01:22:56.334074 24578 external_mini_cluster.cc:1428] Setting key 03334299d2d1e25c6168e71db8c5d1ab
I20260812 01:22:56.335050 24578 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 01:22:56.339161 21649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:56.336460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33362 (local address 127.24.0.190:44097)
0812 01:22:56.336537 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0812 01:22:56.336539 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:22:56.336617 (+ 78us) server_negotiation.cc:407] Connection header received
0812 01:22:56.336641 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:22:56.336642 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:22:56.336651 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:22:56.336700 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:22:56.336836 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:56.337291 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:56.338039 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:56.338223 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:56.338279 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:22:56.338857 (+ 578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:22:56.338858 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:22:56.338858 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:22:56.338862 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:22:56.338873 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:22:56.338876 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:22:56.338918 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0812 01:22:56.338969 (+ 51us) server_negotiation.cc:299] Negotiation successful
0812 01:22:56.339046 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 01:22:56.340749 21582 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33362:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 01:22:56.342237 21582 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 01:22:56.348927 22353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:56.345096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51016 (local address 127.24.0.129:34497)
0812 01:22:56.345397 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0812 01:22:56.345401 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 01:22:56.345410 (+ 9us) server_negotiation.cc:407] Connection header received
0812 01:22:56.345448 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:22:56.345450 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:22:56.345465 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:22:56.345571 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:22:56.345673 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:56.346765 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:56.347988 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:56.348536 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:56.348624 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:22:56.348717 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0812 01:22:56.348804 (+ 87us) server_negotiation.cc:299] Negotiation successful
0812 01:22:56.348841 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":257,"thread_start_us":90,"threads_started":1}
I20260812 01:22:56.349015 22346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:56.344565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.131:34023 (local address 127.0.0.1:54122)
0812 01:22:56.344773 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0812 01:22:56.344814 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 01:22:56.344900 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:22:56.345410 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:22:56.345411 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:22:56.345413 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:22:56.345448 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:22:56.345449 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:22:56.346726 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:22:56.346727 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:22:56.347788 (+ 1061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:22:56.347791 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:22:56.348742 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:22:56.348743 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:22:56.348806 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:22:56.348810 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 01:22:56.348842 (+ 32us) client_negotiation.cc:240] Negotiation successful
0812 01:22:56.348879 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"mutex_wait_us":12,"thread_start_us":81,"threads_started":1}
I20260812 01:22:56.349068 22350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:56.344609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.130:38947 (local address 127.0.0.1:41254)
0812 01:22:56.344963 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0812 01:22:56.344975 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 01:22:56.345028 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:22:56.345334 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:22:56.345337 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:22:56.345339 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:22:56.345401 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:22:56.345403 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:22:56.346460 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:22:56.346461 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:22:56.347830 (+ 1369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:22:56.347833 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:22:56.348826 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:22:56.348827 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:22:56.348878 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:22:56.348881 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 01:22:56.348936 (+ 55us) client_negotiation.cc:240] Negotiation successful
0812 01:22:56.348973 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":83,"threads_started":1}
I20260812 01:22:56.348953 22345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:56.344797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.129:34497 (local address 127.0.0.1:51016)
0812 01:22:56.345084 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0812 01:22:56.345093 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 01:22:56.345138 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:22:56.345568 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:22:56.345569 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:22:56.345570 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:22:56.345597 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:22:56.345598 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:22:56.346864 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:22:56.346865 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:22:56.347928 (+ 1063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:22:56.347930 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:22:56.348624 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:22:56.348625 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:22:56.348700 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:22:56.348713 (+ 13us) client_negotiation.cc:769] Sending connection context
0812 01:22:56.348789 (+ 76us) client_negotiation.cc:240] Negotiation successful
0812 01:22:56.348838 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":102,"threads_started":1}
I20260812 01:22:56.349309 22351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:56.344789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41254 (local address 127.24.0.130:38947)
0812 01:22:56.345014 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0812 01:22:56.345019 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 01:22:56.345032 (+ 13us) server_negotiation.cc:407] Connection header received
0812 01:22:56.345160 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:22:56.345164 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:22:56.345182 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:22:56.345254 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:22:56.345488 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:56.346370 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:56.347916 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:56.348634 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:56.348709 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:22:56.348794 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 01:22:56.349180 (+ 386us) server_negotiation.cc:299] Negotiation successful
0812 01:22:56.349223 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":113,"threads_started":1}
I20260812 01:22:56.349370 22347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:56.344684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54122 (local address 127.24.0.131:34023)
0812 01:22:56.345233 (+ 549us) server_negotiation.cc:206] Beginning negotiation
0812 01:22:56.345237 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 01:22:56.345246 (+ 9us) server_negotiation.cc:407] Connection header received
0812 01:22:56.345282 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:22:56.345284 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:22:56.345301 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:22:56.345409 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:22:56.345507 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:56.346625 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:56.347893 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:56.348634 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:56.348999 (+ 365us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:22:56.349086 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 01:22:56.349114 (+ 28us) server_negotiation.cc:299] Negotiation successful
0812 01:22:56.349273 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":513,"thread_start_us":86,"threads_started":1}
I20260812 01:22:56.350066 21999 tablet_service.cc:1467] Processing CreateTablet for tablet 3ffbc7169ef64b3c990602663a0425a9 (DEFAULT_TABLE table=test-table [id=7db1fe5ad6514697baa4b5bb1e3f3e6c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:22:56.350347 21786 tablet_service.cc:1467] Processing CreateTablet for tablet 3ffbc7169ef64b3c990602663a0425a9 (DEFAULT_TABLE table=test-table [id=7db1fe5ad6514697baa4b5bb1e3f3e6c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:22:56.350433 21999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ffbc7169ef64b3c990602663a0425a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:22:56.350667 21786 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ffbc7169ef64b3c990602663a0425a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:22:56.352495 22225 tablet_service.cc:1467] Processing CreateTablet for tablet 3ffbc7169ef64b3c990602663a0425a9 (DEFAULT_TABLE table=test-table [id=7db1fe5ad6514697baa4b5bb1e3f3e6c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:22:56.352824 22225 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ffbc7169ef64b3c990602663a0425a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:22:56.353297 22354 tablet_bootstrap.cc:492] T 3ffbc7169ef64b3c990602663a0425a9 P ae7b2fcbdede4e71a5546877f6850599: Bootstrap starting.
I20260812 01:22:56.353632 22355 tablet_bootstrap.cc:492] T 3ffbc7169ef64b3c990602663a0425a9 P 69ca216d655348c29f93945a75dd4783: Bootstrap starting.
I20260812 01:22:56.354118 22354 tablet_bootstrap.cc:654] T 3ffbc7169ef64b3c990602663a0425a9 P ae7b2fcbdede4e71a5546877f6850599: Neither blocks nor log segments found. Creating new log.
I20260812 01:22:56.354285 22355 tablet_bootstrap.cc:654] T 3ffbc7169ef64b3c990602663a0425a9 P 69ca216d655348c29f93945a75dd4783: Neither blocks nor log segments found. Creating new log.
I20260812 01:22:56.354485 22354 log.cc:824] T 3ffbc7169ef64b3c990602663a0425a9 P ae7b2fcbdede4e71a5546877f6850599: Log is configured to *not* fsync() on all Append() calls
I20260812 01:22:56.354617 22355 log.cc:824] T 3ffbc7169ef64b3c990602663a0425a9 P 69ca216d655348c29f93945a75dd4783: Log is configured to *not* fsync() on all Append() calls
I20260812 01:22:56.355274 22354 tablet_bootstrap.cc:492] T 3ffbc7169ef64b3c990602663a0425a9 P ae7b2fcbdede4e71a5546877f6850599: No bootstrap required, opened a new log
I20260812 01:22:56.355329 22354 ts_tablet_manager.cc:1397] T 3ffbc7169ef64b3c990602663a0425a9 P ae7b2fcbdede4e71a5546877f6850599: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 01:22:56.355515 22355 tablet_bootstrap.cc:492] T 3ffbc7169ef64b3c990602663a0425a9 P 69ca216d655348c29f93945a75dd4783: No bootstrap required, opened a new log
I20260812 01:22:56.355655 22355 ts_tablet_manager.cc:1397] T 3ffbc7169ef64b3c990602663a0425a9 P 69ca216d655348c29f93945a75dd4783: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 01:22:56.356086 22361 tablet_bootstrap.cc:492] T 3ffbc7169ef64b3c990602663a0425a9 P 96427713b1344b6189e61adce4036328: Bootstrap starting.
I20260812 01:22:56.356756 22361 tablet_bootstrap.cc:654] T 3ffbc7169ef64b3c990602663a0425a9 P 96427713b1344b6189e61adce4036328: Neither blocks nor log segments found. Creating new log.
I20260812 01:22:56.357081 22361 log.cc:824] T 3ffbc7169ef64b3c990602663a0425a9 P 96427713b1344b6189e61adce4036328: Log is configured to *not* fsync() on all Append() calls
I20260812 01:22:56.357851 22361 tablet_bootstrap.cc:492] T 3ffbc7169ef64b3c990602663a0425a9 P 96427713b1344b6189e61adce4036328: No bootstrap required, opened a new log
I20260812 01:22:56.357923 22361 ts_tablet_manager.cc:1397] T 3ffbc7169ef64b3c990602663a0425a9 P 96427713b1344b6189e61adce4036328: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 01:22:56.358043 22355 raft_consensus.cc:348] T 3ffbc7169ef64b3c990602663a0425a9 P 69ca216d655348c29f93945a75dd4783 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96427713b1344b6189e61adce4036328" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34023 } } peers { permanent_uuid: "69ca216d655348c29f93945a75dd4783" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 34497 } } peers { permanent_uuid: "ae7b2fcbdede4e71a5546877f6850599" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 38947 } }
I20260812 01:22:56.358031 22354 raft_consensus.cc:348] T 3ffbc7169ef64b3c990602663a0425a9 P ae7b2fcbdede4e71a5546877f6850599 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96427713b1344b6189e61adce4036328" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34023 } } peers { permanent_uuid: "69ca216d655348c29f93945a75dd4783" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 34497 } } peers { permanent_uuid: "ae7b2fcbdede4e71a5546877f6850599" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 38947 } }
I20260812 01:22:56.358188 22354 raft_consensus.cc:374] T 3ffbc7169ef64b3c990602663a0425a9 P ae7b2fcbdede4e71a5546877f6850599 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:22:56.358189 22355 raft_consensus.cc:374] T 3ffbc7169ef64b3c990602663a0425a9 P 69ca216d655348c29f93945a75dd4783 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:22:56.358233 22355 raft_consensus.cc:729] T 3ffbc7169ef64b3c990602663a0425a9 P 69ca216d655348c29f93945a75dd4783 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69ca216d655348c29f93945a75dd4783, State: Initialized, Role: FOLLOWER
I20260812 01:22:56.358234 22354 raft_consensus.cc:729] T 3ffbc7169ef64b3c990602663a0425a9 P ae7b2fcbdede4e71a5546877f6850599 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae7b2fcbdede4e71a5546877f6850599, State: Initialized, Role: FOLLOWER
I20260812 01:22:56.358358 22355 consensus_queue.cc:260] T 3ffbc7169ef64b3c990602663a0425a9 P 69ca216d655348c29f93945a75dd4783 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96427713b1344b6189e61adce4036328" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34023 } } peers { permanent_uuid: "69ca216d655348c29f93945a75dd4783" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 34497 } } peers { permanent_uuid: "ae7b2fcbdede4e71a5546877f6850599" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 38947 } }
I20260812 01:22:56.358376 22354 consensus_queue.cc:260] T 3ffbc7169ef64b3c990602663a0425a9 P ae7b2fcbdede4e71a5546877f6850599 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96427713b1344b6189e61adce4036328" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34023 } } peers { permanent_uuid: "69ca216d655348c29f93945a75dd4783" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 34497 } } peers { permanent_uuid: "ae7b2fcbdede4e71a5546877f6850599" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 38947 } }
I20260812 01:22:56.358644 22355 ts_tablet_manager.cc:1428] T 3ffbc7169ef64b3c990602663a0425a9 P 69ca216d655348c29f93945a75dd4783: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 01:22:56.358649 22354 ts_tablet_manager.cc:1428] T 3ffbc7169ef64b3c990602663a0425a9 P ae7b2fcbdede4e71a5546877f6850599: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260812 01:22:56.358664 21893 heartbeater.cc:503] Master 127.24.0.190:44097 was elected leader, sending a full tablet report...
I20260812 01:22:56.358775 22111 heartbeater.cc:503] Master 127.24.0.190:44097 was elected leader, sending a full tablet report...
I20260812 01:22:56.360615 22361 raft_consensus.cc:348] T 3ffbc7169ef64b3c990602663a0425a9 P 96427713b1344b6189e61adce4036328 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96427713b1344b6189e61adce4036328" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34023 } } peers { permanent_uuid: "69ca216d655348c29f93945a75dd4783" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 34497 } } peers { permanent_uuid: "ae7b2fcbdede4e71a5546877f6850599" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 38947 } }
I20260812 01:22:56.360736 22361 raft_consensus.cc:374] T 3ffbc7169ef64b3c990602663a0425a9 P 96427713b1344b6189e61adce4036328 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:22:56.360759 22361 raft_consensus.cc:729] T 3ffbc7169ef64b3c990602663a0425a9 P 96427713b1344b6189e61adce4036328 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96427713b1344b6189e61adce4036328, State: Initialized, Role: FOLLOWER
I20260812 01:22:56.360857 22361 consensus_queue.cc:260] T 3ffbc7169ef64b3c990602663a0425a9 P 96427713b1344b6189e61adce4036328 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96427713b1344b6189e61adce4036328" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34023 } } peers { permanent_uuid: "69ca216d655348c29f93945a75dd4783" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 34497 } } peers { permanent_uuid: "ae7b2fcbdede4e71a5546877f6850599" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 38947 } }
I20260812 01:22:56.361058 22361 ts_tablet_manager.cc:1428] T 3ffbc7169ef64b3c990602663a0425a9 P 96427713b1344b6189e61adce4036328: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 01:22:56.361178 22329 heartbeater.cc:503] Master 127.24.0.190:44097 was elected leader, sending a full tablet report...
W20260812 01:22:56.449194 22112 tablet.cc:2369] T 3ffbc7169ef64b3c990602663a0425a9 P ae7b2fcbdede4e71a5546877f6850599: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 01:22:56.569916 21894 tablet.cc:2369] T 3ffbc7169ef64b3c990602663a0425a9 P 69ca216d655348c29f93945a75dd4783: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 01:22:56.576898 22330 tablet.cc:2369] T 3ffbc7169ef64b3c990602663a0425a9 P 96427713b1344b6189e61adce4036328: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 01:22:56.579139 22364 raft_consensus.cc:482] T 3ffbc7169ef64b3c990602663a0425a9 P 69ca216d655348c29f93945a75dd4783 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 01:22:56.579265 22364 raft_consensus.cc:504] T 3ffbc7169ef64b3c990602663a0425a9 P 69ca216d655348c29f93945a75dd4783 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96427713b1344b6189e61adce4036328" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34023 } } peers { permanent_uuid: "69ca216d655348c29f93945a75dd4783" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 34497 } } peers { permanent_uuid: "ae7b2fcbdede4e71a5546877f6850599" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 38947 } }
I20260812 01:22:56.579712 22364 leader_election.cc:288] T 3ffbc7169ef64b3c990602663a0425a9 P 69ca216d655348c29f93945a75dd4783 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 96427713b1344b6189e61adce4036328 (127.24.0.131:34023), ae7b2fcbdede4e71a5546877f6850599 (127.24.0.130:38947)
I20260812 01:22:56.582871 22347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:56.579995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.129:46715 (local address 127.24.0.131:34023)
0812 01:22:56.580051 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 01:22:56.580053 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:22:56.580131 (+ 78us) server_negotiation.cc:407] Connection header received
0812 01:22:56.580239 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:22:56.580241 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:22:56.580253 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:22:56.580310 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:22:56.580495 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:56.581039 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:56.582021 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:56.582475 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:56.582543 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:22:56.582551 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 01:22:56.582747 (+ 196us) server_negotiation.cc:299] Negotiation successful
0812 01:22:56.582796 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260812 01:22:56.582876 22367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:56.579913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.131:34023 (local address 127.24.0.129:46715)
0812 01:22:56.580050 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0812 01:22:56.580061 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 01:22:56.580163 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:22:56.580337 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:22:56.580338 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:22:56.580341 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:22:56.580423 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:22:56.580425 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:22:56.581156 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:22:56.581157 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:22:56.581930 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:22:56.581932 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:22:56.582566 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:22:56.582567 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:22:56.582667 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:22:56.582676 (+ 9us) client_negotiation.cc:769] Sending connection context
0812 01:22:56.582720 (+ 44us) client_negotiation.cc:240] Negotiation successful
0812 01:22:56.582778 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":70,"threads_started":1}
I20260812 01:22:56.582917 22351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:56.580026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.129:34019 (local address 127.24.0.130:38947)
0812 01:22:56.580105 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0812 01:22:56.580107 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:22:56.580154 (+ 47us) server_negotiation.cc:407] Connection header received
0812 01:22:56.580252 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:22:56.580253 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:22:56.580264 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:22:56.580347 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:22:56.580570 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:56.581018 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:56.582162 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:56.582529 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:56.582602 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:22:56.582612 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 01:22:56.582801 (+ 189us) server_negotiation.cc:299] Negotiation successful
0812 01:22:56.582842 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 01:22:56.582978 22368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:56.579916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.130:38947 (local address 127.24.0.129:34019)
0812 01:22:56.580088 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0812 01:22:56.580100 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 01:22:56.580168 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:22:56.580373 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:22:56.580374 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:22:56.580376 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:22:56.580479 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:22:56.580482 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:22:56.581108 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:22:56.581109 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:22:56.582015 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:22:56.582017 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:22:56.582620 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:22:56.582622 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:22:56.582694 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:22:56.582698 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 01:22:56.582779 (+ 81us) client_negotiation.cc:240] Negotiation successful
0812 01:22:56.582815 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"mutex_wait_us":48,"thread_start_us":76,"threads_started":1}
I20260812 01:22:56.583206 22255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ffbc7169ef64b3c990602663a0425a9" candidate_uuid: "69ca216d655348c29f93945a75dd4783" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96427713b1344b6189e61adce4036328" is_pre_election: true
I20260812 01:22:56.583370 22255 raft_consensus.cc:2454] T 3ffbc7169ef64b3c990602663a0425a9 P 96427713b1344b6189e61adce4036328 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 69ca216d655348c29f93945a75dd4783 in term 0.
I20260812 01:22:56.583415 22031 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ffbc7169ef64b3c990602663a0425a9" candidate_uuid: "69ca216d655348c29f93945a75dd4783" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae7b2fcbdede4e71a5546877f6850599" is_pre_election: true
I20260812 01:22:56.583576 22031 raft_consensus.cc:2454] T 3ffbc7169ef64b3c990602663a0425a9 P ae7b2fcbdede4e71a5546877f6850599 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 69ca216d655348c29f93945a75dd4783 in term 0.
I20260812 01:22:56.583639 21712 leader_election.cc:302] T 3ffbc7169ef64b3c990602663a0425a9 P 69ca216d655348c29f93945a75dd4783 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 69ca216d655348c29f93945a75dd4783, 96427713b1344b6189e61adce4036328; no voters:
I20260812 01:22:56.583767 22364 raft_consensus.cc:2781] T 3ffbc7169ef64b3c990602663a0425a9 P 69ca216d655348c29f93945a75dd4783 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 01:22:56.583856 22364 raft_consensus.cc:482] T 3ffbc7169ef64b3c990602663a0425a9 P 69ca216d655348c29f93945a75dd4783 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 01:22:56.583886 22364 raft_consensus.cc:3037] T 3ffbc7169ef64b3c990602663a0425a9 P 69ca216d655348c29f93945a75dd4783 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:22:56.584726 22364 raft_consensus.cc:504] T 3ffbc7169ef64b3c990602663a0425a9 P 69ca216d655348c29f93945a75dd4783 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96427713b1344b6189e61adce4036328" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34023 } } peers { permanent_uuid: "69ca216d655348c29f93945a75dd4783" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 34497 } } peers { permanent_uuid: "ae7b2fcbdede4e71a5546877f6850599" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 38947 } }
I20260812 01:22:56.584882 22364 leader_election.cc:288] T 3ffbc7169ef64b3c990602663a0425a9 P 69ca216d655348c29f93945a75dd4783 [CANDIDATE]: Term 1 election: Requested vote from peers 96427713b1344b6189e61adce4036328 (127.24.0.131:34023), ae7b2fcbdede4e71a5546877f6850599 (127.24.0.130:38947)
I20260812 01:22:56.585050 22255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ffbc7169ef64b3c990602663a0425a9" candidate_uuid: "69ca216d655348c29f93945a75dd4783" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96427713b1344b6189e61adce4036328"
I20260812 01:22:56.585044 22031 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ffbc7169ef64b3c990602663a0425a9" candidate_uuid: "69ca216d655348c29f93945a75dd4783" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae7b2fcbdede4e71a5546877f6850599"
I20260812 01:22:56.585134 22255 raft_consensus.cc:3037] T 3ffbc7169ef64b3c990602663a0425a9 P 96427713b1344b6189e61adce4036328 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:22:56.585148 22031 raft_consensus.cc:3037] T 3ffbc7169ef64b3c990602663a0425a9 P ae7b2fcbdede4e71a5546877f6850599 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:22:56.585872 22255 raft_consensus.cc:2454] T 3ffbc7169ef64b3c990602663a0425a9 P 96427713b1344b6189e61adce4036328 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 69ca216d655348c29f93945a75dd4783 in term 1.
I20260812 01:22:56.586032 22031 raft_consensus.cc:2454] T 3ffbc7169ef64b3c990602663a0425a9 P ae7b2fcbdede4e71a5546877f6850599 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 69ca216d655348c29f93945a75dd4783 in term 1.
I20260812 01:22:56.586043 21712 leader_election.cc:302] T 3ffbc7169ef64b3c990602663a0425a9 P 69ca216d655348c29f93945a75dd4783 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 69ca216d655348c29f93945a75dd4783, 96427713b1344b6189e61adce4036328; no voters:
I20260812 01:22:56.586130 22364 raft_consensus.cc:2781] T 3ffbc7169ef64b3c990602663a0425a9 P 69ca216d655348c29f93945a75dd4783 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:22:56.586306 22364 raft_consensus.cc:686] T 3ffbc7169ef64b3c990602663a0425a9 P 69ca216d655348c29f93945a75dd4783 [term 1 LEADER]: Becoming Leader. State: Replica: 69ca216d655348c29f93945a75dd4783, State: Running, Role: LEADER
I20260812 01:22:56.586423 22364 consensus_queue.cc:237] T 3ffbc7169ef64b3c990602663a0425a9 P 69ca216d655348c29f93945a75dd4783 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96427713b1344b6189e61adce4036328" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34023 } } peers { permanent_uuid: "69ca216d655348c29f93945a75dd4783" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 34497 } } peers { permanent_uuid: "ae7b2fcbdede4e71a5546877f6850599" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 38947 } }
I20260812 01:22:56.587330 21582 catalog_manager.cc:5495] T 3ffbc7169ef64b3c990602663a0425a9 P 69ca216d655348c29f93945a75dd4783 reported cstate change: term changed from 0 to 1, leader changed from <none> to 69ca216d655348c29f93945a75dd4783 (127.24.0.129). New cstate: current_term: 1 leader_uuid: "69ca216d655348c29f93945a75dd4783" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96427713b1344b6189e61adce4036328" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "69ca216d655348c29f93945a75dd4783" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 34497 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ae7b2fcbdede4e71a5546877f6850599" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 38947 } health_report { overall_health: UNKNOWN } } }
I20260812 01:22:56.632144 22353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:56.629609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51026 (local address 127.24.0.129:34497)
0812 01:22:56.629708 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0812 01:22:56.629710 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:22:56.629721 (+ 11us) server_negotiation.cc:407] Connection header received
0812 01:22:56.629762 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:22:56.629763 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:22:56.629786 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:22:56.629865 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:22:56.629997 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:56.630511 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:56.631313 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:56.631536 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:56.631600 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:22:56.631747 (+ 147us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:22:56.631858 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:22:56.631926 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 01:22:56.631957 (+ 31us) server_negotiation.cc:299] Negotiation successful
0812 01:22:56.631977 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260812 01:22:56.634253 22255 raft_consensus.cc:1261] T 3ffbc7169ef64b3c990602663a0425a9 P 96427713b1344b6189e61adce4036328 [term 1 FOLLOWER]: Refusing update from remote peer 69ca216d655348c29f93945a75dd4783: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:22:56.634251 22031 raft_consensus.cc:1261] T 3ffbc7169ef64b3c990602663a0425a9 P ae7b2fcbdede4e71a5546877f6850599 [term 1 FOLLOWER]: Refusing update from remote peer 69ca216d655348c29f93945a75dd4783: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:22:56.634626 22369 consensus_queue.cc:1035] T 3ffbc7169ef64b3c990602663a0425a9 P 69ca216d655348c29f93945a75dd4783 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96427713b1344b6189e61adce4036328" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 34023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:22:56.634768 22364 consensus_queue.cc:1035] T 3ffbc7169ef64b3c990602663a0425a9 P 69ca216d655348c29f93945a75dd4783 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ae7b2fcbdede4e71a5546877f6850599" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 38947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:22:56.637619 22377 mvcc.cc:205] Tried to move back new op lower bound from 7317494893090648064 to 7317494892898656256. Current Snapshot: MvccSnapshot[applied={T|T < 7317494893090648064}]
I20260812 01:22:56.638391 22376 mvcc.cc:205] Tried to move back new op lower bound from 7317494893090648064 to 7317494892898656256. Current Snapshot: MvccSnapshot[applied={T|T < 7317494893090648064 or (T in {7317494893090648064})}]
I20260812 01:22:56.639799 21582 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33362:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:22:56.639901 21582 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33362:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:22:56.640748 21582 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fe886efefe78449ca8b658168cb324a1: Sending DeleteTablet for 3 replicas of tablet 3ffbc7169ef64b3c990602663a0425a9
I20260812 01:22:56.641142 21999 tablet_service.cc:1514] Processing DeleteTablet for tablet 3ffbc7169ef64b3c990602663a0425a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:22:56 UTC) from {username='slave'} at 127.0.0.1:41254
I20260812 01:22:56.641152 22225 tablet_service.cc:1514] Processing DeleteTablet for tablet 3ffbc7169ef64b3c990602663a0425a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:22:56 UTC) from {username='slave'} at 127.0.0.1:54122
I20260812 01:22:56.641206 21786 tablet_service.cc:1514] Processing DeleteTablet for tablet 3ffbc7169ef64b3c990602663a0425a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:22:56 UTC) from {username='slave'} at 127.0.0.1:51016
I20260812 01:22:56.641410 22397 tablet_replica.cc:330] T 3ffbc7169ef64b3c990602663a0425a9 P 69ca216d655348c29f93945a75dd4783: stopping tablet replica
I20260812 01:22:56.641489 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 21686
I20260812 01:22:56.641507 22397 raft_consensus.cc:2229] T 3ffbc7169ef64b3c990602663a0425a9 P 69ca216d655348c29f93945a75dd4783 [term 1 LEADER]: Raft consensus shutting down.
I20260812 01:22:56.652649 22395 tablet_replica.cc:330] T 3ffbc7169ef64b3c990602663a0425a9 P ae7b2fcbdede4e71a5546877f6850599: stopping tablet replica
I20260812 01:22:56.652844 22395 raft_consensus.cc:2229] T 3ffbc7169ef64b3c990602663a0425a9 P ae7b2fcbdede4e71a5546877f6850599 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 01:22:56.652998 22396 tablet_replica.cc:330] T 3ffbc7169ef64b3c990602663a0425a9 P 96427713b1344b6189e61adce4036328: stopping tablet replica
I20260812 01:22:56.653072 22395 raft_consensus.cc:2258] T 3ffbc7169ef64b3c990602663a0425a9 P ae7b2fcbdede4e71a5546877f6850599 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:22:56.653136 22396 raft_consensus.cc:2229] T 3ffbc7169ef64b3c990602663a0425a9 P 96427713b1344b6189e61adce4036328 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 01:22:56.654248 22396 raft_consensus.cc:2258] T 3ffbc7169ef64b3c990602663a0425a9 P 96427713b1344b6189e61adce4036328 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:22:56.654840 22395 ts_tablet_manager.cc:1905] T 3ffbc7169ef64b3c990602663a0425a9 P ae7b2fcbdede4e71a5546877f6850599: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:22:56.654871 22396 ts_tablet_manager.cc:1905] T 3ffbc7169ef64b3c990602663a0425a9 P 96427713b1344b6189e61adce4036328: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:22:56.656519 22395 ts_tablet_manager.cc:1918] T 3ffbc7169ef64b3c990602663a0425a9 P ae7b2fcbdede4e71a5546877f6850599: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 01:22:56.656620 22395 log.cc:1196] T 3ffbc7169ef64b3c990602663a0425a9 P ae7b2fcbdede4e71a5546877f6850599: Deleting WAL directory at /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786497534195564-24578-0/minicluster-data/ts-1/wal/wals/3ffbc7169ef64b3c990602663a0425a9
I20260812 01:22:56.656873 22395 ts_tablet_manager.cc:1939] T 3ffbc7169ef64b3c990602663a0425a9 P ae7b2fcbdede4e71a5546877f6850599: Deleting consensus metadata
I20260812 01:22:56.657150 22396 ts_tablet_manager.cc:1918] T 3ffbc7169ef64b3c990602663a0425a9 P 96427713b1344b6189e61adce4036328: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 01:22:56.657248 22396 log.cc:1196] T 3ffbc7169ef64b3c990602663a0425a9 P 96427713b1344b6189e61adce4036328: Deleting WAL directory at /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786497534195564-24578-0/minicluster-data/ts-2/wal/wals/3ffbc7169ef64b3c990602663a0425a9
I20260812 01:22:56.657385 21560 catalog_manager.cc:4841] TS ae7b2fcbdede4e71a5546877f6850599 (127.24.0.130:38947): tablet 3ffbc7169ef64b3c990602663a0425a9 (table test-table [id=7db1fe5ad6514697baa4b5bb1e3f3e6c]) successfully deleted
I20260812 01:22:56.657569 22396 ts_tablet_manager.cc:1939] T 3ffbc7169ef64b3c990602663a0425a9 P 96427713b1344b6189e61adce4036328: Deleting consensus metadata
I20260812 01:22:56.658043 21557 catalog_manager.cc:4841] TS 96427713b1344b6189e61adce4036328 (127.24.0.131:34023): tablet 3ffbc7169ef64b3c990602663a0425a9 (table test-table [id=7db1fe5ad6514697baa4b5bb1e3f3e6c]) successfully deleted
W20260812 01:22:56.659876 21557 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.0.129:34497 (error 108)
I20260812 01:22:56.660132 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 21896
I20260812 01:22:56.660487 22350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:56.660344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.129:34497 (local address 127.0.0.1:51030)
0812 01:22:56.660401 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0812 01:22:56.660442 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.129:34497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260812 01:22:56.660602 21557 catalog_manager.cc:4584] TS 69ca216d655348c29f93945a75dd4783 (127.24.0.129:34497): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3ffbc7169ef64b3c990602663a0425a9: Network error: Client connection negotiation failed: client connection to 127.24.0.129:34497: connect: Connection refused (error 111)
I20260812 01:22:56.674239 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 22116
I20260812 01:22:56.692294 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 21535
2026-08-12T01:22:56Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (939 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786497534195564-24578-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 01:22:56 dist-test-slave-t3q3 krb5kdc[22408](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 01:22:56 dist-test-slave-t3q3 krb5kdc[22408](info): set up 2 sockets
Aug 12 01:22:56 dist-test-slave-t3q3 krb5kdc[22408](info): commencing operation
krb5kdc: starting...
W20260812 01:22:57.975524 24578 mini_kdc.cc:121] Time spent starting KDC: real 1.256s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786497534195564-24578-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786497534195564-24578-0/krb5kdc/test-user.keytab.
Aug 12 01:22:58 dist-test-slave-t3q3 krb5kdc[22408](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497778, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T01:22:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:22:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.0.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.0.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.0.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab.
Entry for principal kudu/127.24.0.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab.
I20260812 01:22:58.102464 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786497534195564-24578-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786497534195564-24578-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786497534195564-24578-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.190:43581
--webserver_interface=127.24.0.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab
--principal=kudu/127.24.0.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:45737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.0.190:43581
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:22:58.186583 22421 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:22:58.186822 22421 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:22:58.186861 22421 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:22:58.188239 22421 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:22:58.188308 22421 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:22:58.188338 22421 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:22:58.188366 22421 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:22:58.188393 22421 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:22:58.189869 22421 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:45737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786497534195564-24578-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786497534195564-24578-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.0.190:43581
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab
--principal=kudu/127.24.0.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.190:43581
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.0.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22421
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786497534195564-24578-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:22:58.190169 22421 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:22:58.190395 22421 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:22:58.195242 22431 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:22:58.195223 22427 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:22:58.195497 22421 server_base.cc:1034] running on GCE node
W20260812 01:22:58.195219 22433 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:22:58.195823 22421 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:22:58.196211 22421 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:22:58.197397 22421 hybrid_clock.cc:648] HybridClock initialized: now 1786497778197383 us; error 49 us; skew 500 ppm
Aug 12 01:22:58 dist-test-slave-t3q3 krb5kdc[22408](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497778, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:22:58.200006 22421 init.cc:377] Logged in from keytab as kudu/127.24.0.190@KRBTEST.COM (short username kudu)
I20260812 01:22:58.200512 22421 webserver.cc:460] Webserver started at http://127.24.0.190:38461/ using document root <none> and password file <none>
I20260812 01:22:58.200764 22421 fs_manager.cc:359] Metadata directory not provided
I20260812 01:22:58.200847 22421 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:22:58.201014 22421 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:22:58.202075 22421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786497534195564-24578-0/minicluster-data/master-0/data/instance:
uuid: "6fd156381b734def8eeac63b70845f2f"
format_stamp: "Formatted at 2026-08-12 01:22:58 on dist-test-slave-t3q3"
server_key: "7827578272c6ac6bb42269a93963c998"
server_key_iv: "09e3452b6d2caa0b78d117296051e6c1"
server_key_version: "encryptionkey@0"
I20260812 01:22:58.202447 22421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786497534195564-24578-0/minicluster-data/master-0/wal/instance:
uuid: "6fd156381b734def8eeac63b70845f2f"
format_stamp: "Formatted at 2026-08-12 01:22:58 on dist-test-slave-t3q3"
server_key: "7827578272c6ac6bb42269a93963c998"
server_key_iv: "09e3452b6d2caa0b78d117296051e6c1"
server_key_version: "encryptionkey@0"
I20260812 01:22:58.203804 22421 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260812 01:22:58.204643 22441 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:22:58.204824 22421 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 01:22:58.204926 22421 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786497534195564-24578-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786497534195564-24578-0/minicluster-data/master-0/wal
uuid: "6fd156381b734def8eeac63b70845f2f"
format_stamp: "Formatted at 2026-08-12 01:22:58 on dist-test-slave-t3q3"
server_key: "7827578272c6ac6bb42269a93963c998"
server_key_iv: "09e3452b6d2caa0b78d117296051e6c1"
server_key_version: "encryptionkey@0"
I20260812 01:22:58.205018 22421 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786497534195564-24578-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786497534195564-24578-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786497534195564-24578-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:22:58.223301 22421 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:22:58.225337 22421 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:22:58.225517 22421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:22:58.229592 22421 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.190:43581
I20260812 01:22:58.229630 22524 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.190:43581 every 8 connection(s)
I20260812 01:22:58.230005 22421 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
I20260812 01:22:58.231001 22525 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:22:58.233562 22525 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6fd156381b734def8eeac63b70845f2f: Bootstrap starting.
I20260812 01:22:58.234225 22525 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6fd156381b734def8eeac63b70845f2f: Neither blocks nor log segments found. Creating new log.
I20260812 01:22:58.234556 22525 log.cc:824] T 00000000000000000000000000000000 P 6fd156381b734def8eeac63b70845f2f: Log is configured to *not* fsync() on all Append() calls
I20260812 01:22:58.235358 22525 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6fd156381b734def8eeac63b70845f2f: No bootstrap required, opened a new log
I20260812 01:22:58.237254 22525 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6fd156381b734def8eeac63b70845f2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fd156381b734def8eeac63b70845f2f" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 43581 } }
I20260812 01:22:58.237403 22525 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6fd156381b734def8eeac63b70845f2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:22:58.237427 22525 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6fd156381b734def8eeac63b70845f2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fd156381b734def8eeac63b70845f2f, State: Initialized, Role: FOLLOWER
I20260812 01:22:58.237620 22525 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6fd156381b734def8eeac63b70845f2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fd156381b734def8eeac63b70845f2f" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 43581 } }
I20260812 01:22:58.237707 22525 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6fd156381b734def8eeac63b70845f2f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:22:58.237751 22525 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6fd156381b734def8eeac63b70845f2f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:22:58.237805 22525 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6fd156381b734def8eeac63b70845f2f [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:22:58.237991 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 22421
I20260812 01:22:58.238107 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786497534195564-24578-0/minicluster-data/master-0/wal/instance
I20260812 01:22:58.238312 24578 external_mini_cluster.cc:1428] Setting key 520d7da858ec86419e0843831349e3b2
I20260812 01:22:58.238584 22525 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6fd156381b734def8eeac63b70845f2f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fd156381b734def8eeac63b70845f2f" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 43581 } }
I20260812 01:22:58.238714 22525 leader_election.cc:302] T 00000000000000000000000000000000 P 6fd156381b734def8eeac63b70845f2f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6fd156381b734def8eeac63b70845f2f; no voters:
I20260812 01:22:58.238907 22525 leader_election.cc:288] T 00000000000000000000000000000000 P 6fd156381b734def8eeac63b70845f2f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 01:22:58.238947 22530 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6fd156381b734def8eeac63b70845f2f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:22:58.239171 22530 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6fd156381b734def8eeac63b70845f2f [term 1 LEADER]: Becoming Leader. State: Replica: 6fd156381b734def8eeac63b70845f2f, State: Running, Role: LEADER
I20260812 01:22:58.239305 22525 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6fd156381b734def8eeac63b70845f2f [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:22:58.239318 22530 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6fd156381b734def8eeac63b70845f2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fd156381b734def8eeac63b70845f2f" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 43581 } }
I20260812 01:22:58.242091 22534 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6fd156381b734def8eeac63b70845f2f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6fd156381b734def8eeac63b70845f2f. Latest consensus state: current_term: 1 leader_uuid: "6fd156381b734def8eeac63b70845f2f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fd156381b734def8eeac63b70845f2f" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 43581 } } }
I20260812 01:22:58.242197 22534 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6fd156381b734def8eeac63b70845f2f [sys.catalog]: This master's current role is: LEADER
I20260812 01:22:58.242383 22532 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6fd156381b734def8eeac63b70845f2f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6fd156381b734def8eeac63b70845f2f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fd156381b734def8eeac63b70845f2f" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 43581 } } }
I20260812 01:22:58.242467 22532 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6fd156381b734def8eeac63b70845f2f [sys.catalog]: This master's current role is: LEADER
W20260812 01:22:58.241505 22550 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6fd156381b734def8eeac63b70845f2f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 01:22:58.242544 22550 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 01:22:58.242615 22554 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:22:58.242800 22554 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:22:58.245486 22554 catalog_manager.cc:1347] Generated new cluster ID: ffa1904175b348779da26cc5378e32a1
I20260812 01:22:58.245548 22554 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 01:22:58 dist-test-slave-t3q3 krb5kdc[22408](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497778, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
I20260812 01:22:58.250643 22531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:58.238818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50184 (local address 127.24.0.190:43581)
0812 01:22:58.239012 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0812 01:22:58.239022 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 01:22:58.239062 (+ 40us) server_negotiation.cc:407] Connection header received
0812 01:22:58.239201 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:22:58.239209 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:22:58.239268 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:22:58.239497 (+ 229us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:22:58.240151 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:58.241643 (+ 1492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:58.242761 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:58.243374 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:58.243480 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:22:58.246264 (+ 2784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:22:58.246268 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:22:58.246268 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:22:58.246279 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:22:58.248909 (+ 2630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:22:58.249348 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:22:58.249353 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:22:58.249354 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:22:58.249466 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:22:58.249725 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:22:58.249726 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:22:58.249726 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:22:58.249998 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:22:58.250180 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0812 01:22:58.250305 (+ 125us) server_negotiation.cc:299] Negotiation successful
0812 01:22:58.250465 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":81,"threads_started":1}
I20260812 01:22:58.259943 22554 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:22:58.260686 22554 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:22:58.263866 22554 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6fd156381b734def8eeac63b70845f2f: Generated new TSK 0
I20260812 01:22:58.264052 22554 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.0.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.0.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.0.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.129.keytab.
Entry for principal kudu/127.24.0.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.129.keytab.
I20260812 01:22:58.299563 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786497534195564-24578-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786497534195564-24578-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786497534195564-24578-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786497534195564-24578-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.0.129:0
--local_ip_for_outbound_sockets=127.24.0.129
--webserver_interface=127.24.0.129
--webserver_port=0
--tserver_master_addrs=127.24.0.190:43581
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.129.keytab
--principal=kudu/127.24.0.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:45737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:22:58.387629 22558 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:22:58.387879 22558 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:22:58.387920 22558 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:22:58.389135 22558 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:22:58.389206 22558 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:22:58.389250 22558 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.0.129
I20260812 01:22:58.390743 22558 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:45737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786497534195564-24578-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786497534195564-24578-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.129.keytab
--principal=kudu/127.24.0.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.0.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786497534195564-24578-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.0.129
--webserver_port=0
--tserver_master_addrs=127.24.0.190:43581
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22558
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.0.129
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786497534195564-24578-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:22:58.391003 22558 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:22:58.391253 22558 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:22:58.396548 22566 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:22:58.396620 22558 server_base.cc:1034] running on GCE node
W20260812 01:22:58.396556 22569 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:22:58.396556 22567 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:22:58.396976 22558 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:22:58.397272 22558 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:22:58.398459 22558 hybrid_clock.cc:648] HybridClock initialized: now 1786497778398439 us; error 56 us; skew 500 ppm
Aug 12 01:22:58 dist-test-slave-t3q3 krb5kdc[22408](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497778, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:22:58.401197 22558 init.cc:377] Logged in from keytab as kudu/127.24.0.129@KRBTEST.COM (short username kudu)
I20260812 01:22:58.401633 22558 webserver.cc:460] Webserver started at http://127.24.0.129:38647/ using document root <none> and password file <none>
I20260812 01:22:58.401862 22558 fs_manager.cc:359] Metadata directory not provided
I20260812 01:22:58.401932 22558 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:22:58.402088 22558 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:22:58.403167 22558 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786497534195564-24578-0/minicluster-data/ts-0/data/instance:
uuid: "0c20a859489745fbb1f178c0c455885a"
format_stamp: "Formatted at 2026-08-12 01:22:58 on dist-test-slave-t3q3"
server_key: "9171a0b9d7f05fc5cfe7c1fc7ab68c64"
server_key_iv: "1ebb3d4305be6e98314d28078deb9041"
server_key_version: "encryptionkey@0"
I20260812 01:22:58.403548 22558 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786497534195564-24578-0/minicluster-data/ts-0/wal/instance:
uuid: "0c20a859489745fbb1f178c0c455885a"
format_stamp: "Formatted at 2026-08-12 01:22:58 on dist-test-slave-t3q3"
server_key: "9171a0b9d7f05fc5cfe7c1fc7ab68c64"
server_key_iv: "1ebb3d4305be6e98314d28078deb9041"
server_key_version: "encryptionkey@0"
I20260812 01:22:58.404997 22558 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 01:22:58.405783 22579 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:22:58.406014 22558 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:22:58.406138 22558 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786497534195564-24578-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786497534195564-24578-0/minicluster-data/ts-0/wal
uuid: "0c20a859489745fbb1f178c0c455885a"
format_stamp: "Formatted at 2026-08-12 01:22:58 on dist-test-slave-t3q3"
server_key: "9171a0b9d7f05fc5cfe7c1fc7ab68c64"
server_key_iv: "1ebb3d4305be6e98314d28078deb9041"
server_key_version: "encryptionkey@0"
I20260812 01:22:58.406217 22558 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786497534195564-24578-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786497534195564-24578-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786497534195564-24578-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:22:58.416880 22558 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:22:58.418913 22558 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:22:58.419085 22558 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:22:58.419332 22558 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:22:58.419648 22558 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:22:58.419704 22558 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:22:58.419740 22558 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:22:58.419770 22558 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:22:58.425904 22558 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.129:38905
I20260812 01:22:58.425956 22757 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.129:38905 every 8 connection(s)
I20260812 01:22:58.426276 22558 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786497534195564-24578-0/minicluster-data/ts-0/data/info.pb
I20260812 01:22:58.426750 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 22558
I20260812 01:22:58.426885 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786497534195564-24578-0/minicluster-data/ts-0/wal/instance
I20260812 01:22:58.427135 24578 external_mini_cluster.cc:1428] Setting key bb5b8a93fdda75efe5cdebd6509ca64e
Aug 12 01:22:58 dist-test-slave-t3q3 krb5kdc[22408](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497778, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.129@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.24.0.130@KRBTEST.COM; defaulting to no policy
I20260812 01:22:58.437645 22531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:58.427061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.129:53145 (local address 127.24.0.190:43581)
0812 01:22:58.427140 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0812 01:22:58.427143 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:22:58.427366 (+ 223us) server_negotiation.cc:407] Connection header received
0812 01:22:58.428046 (+ 680us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:22:58.428047 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:22:58.428094 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:22:58.428166 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:22:58.429117 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:58.429893 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:58.431385 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:58.431581 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:58.431638 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:22:58.434394 (+ 2756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:22:58.434396 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:22:58.434396 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:22:58.434405 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:22:58.435997 (+ 1592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:22:58.436507 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:22:58.436508 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:22:58.436509 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:22:58.436579 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:22:58.436889 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:22:58.436890 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:22:58.436890 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:22:58.437025 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:22:58.437115 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 01:22:58.437440 (+ 325us) server_negotiation.cc:299] Negotiation successful
0812 01:22:58.437550 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 01:22:58.437862 22763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:58.427047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:43581 (local address 127.24.0.129:53145)
0812 01:22:58.427242 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0812 01:22:58.427283 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 01:22:58.427925 (+ 642us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:22:58.428243 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:22:58.428247 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:22:58.428830 (+ 583us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:22:58.429015 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:22:58.429019 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:22:58.429992 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:22:58.429993 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:22:58.431291 (+ 1298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:22:58.431294 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:22:58.431671 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:22:58.431672 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:22:58.431821 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:22:58.432498 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:22:58.432510 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:22:58.434279 (+ 1769us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:22:58.436142 (+ 1863us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:22:58.436151 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:22:58.436153 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:22:58.436414 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:22:58.436659 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:22:58.436660 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:22:58.436661 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:22:58.436809 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:22:58.437106 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:22:58.437115 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:22:58.437269 (+ 154us) client_negotiation.cc:769] Sending connection context
0812 01:22:58.437397 (+ 128us) client_negotiation.cc:240] Negotiation successful
0812 01:22:58.437571 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":106,"threads_started":1}
I20260812 01:22:58.438397 22758 heartbeater.cc:347] Connected to a master server at 127.24.0.190:43581
I20260812 01:22:58.438484 22758 heartbeater.cc:464] Registering TS with master...
I20260812 01:22:58.438769 22758 heartbeater.cc:511] Master 127.24.0.190:43581 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.0.130@KRBTEST.COM" created.
I20260812 01:22:58.439402 22461 ts_manager.cc:194] Registered new tserver with Master: 0c20a859489745fbb1f178c0c455885a (127.24.0.129:38905)
I20260812 01:22:58.440105 22461 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.0.129@KRBTEST.COM'} at 127.24.0.129:53145
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.0.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.130.keytab.
Entry for principal kudu/127.24.0.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.130.keytab.
I20260812 01:22:58.452961 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786497534195564-24578-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786497534195564-24578-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786497534195564-24578-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786497534195564-24578-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.0.130:0
--local_ip_for_outbound_sockets=127.24.0.130
--webserver_interface=127.24.0.130
--webserver_port=0
--tserver_master_addrs=127.24.0.190:43581
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.130.keytab
--principal=kudu/127.24.0.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:45737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:22:58.535789 22769 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:22:58.535993 22769 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:22:58.536016 22769 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:22:58.537189 22769 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:22:58.537236 22769 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:22:58.537262 22769 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.0.130
I20260812 01:22:58.538765 22769 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:45737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786497534195564-24578-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786497534195564-24578-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.130.keytab
--principal=kudu/127.24.0.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.0.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786497534195564-24578-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.0.130
--webserver_port=0
--tserver_master_addrs=127.24.0.190:43581
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22769
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.0.130
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786497534195564-24578-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:22:58.539018 22769 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:22:58.539254 22769 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:22:58.544341 22779 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:22:58.544363 22783 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:22:58.544551 22769 server_base.cc:1034] running on GCE node
W20260812 01:22:58.544346 22781 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:22:58.544776 22769 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:22:58.545064 22769 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:22:58.546229 22769 hybrid_clock.cc:648] HybridClock initialized: now 1786497778546217 us; error 28 us; skew 500 ppm
Aug 12 01:22:58 dist-test-slave-t3q3 krb5kdc[22408](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497778, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:22:58.554405 22769 init.cc:377] Logged in from keytab as kudu/127.24.0.130@KRBTEST.COM (short username kudu)
I20260812 01:22:58.554927 22769 webserver.cc:460] Webserver started at http://127.24.0.130:34471/ using document root <none> and password file <none>
I20260812 01:22:58.555181 22769 fs_manager.cc:359] Metadata directory not provided
I20260812 01:22:58.555253 22769 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:22:58.555408 22769 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:22:58.556552 22769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786497534195564-24578-0/minicluster-data/ts-1/data/instance:
uuid: "0d7fb8d2a0fa4826a77b342b02f4cf14"
format_stamp: "Formatted at 2026-08-12 01:22:58 on dist-test-slave-t3q3"
server_key: "10b3f14a9005ca031070cff31e1760ea"
server_key_iv: "81750f16e7daa621a902beae04057a0d"
server_key_version: "encryptionkey@0"
I20260812 01:22:58.556926 22769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786497534195564-24578-0/minicluster-data/ts-1/wal/instance:
uuid: "0d7fb8d2a0fa4826a77b342b02f4cf14"
format_stamp: "Formatted at 2026-08-12 01:22:58 on dist-test-slave-t3q3"
server_key: "10b3f14a9005ca031070cff31e1760ea"
server_key_iv: "81750f16e7daa621a902beae04057a0d"
server_key_version: "encryptionkey@0"
I20260812 01:22:58.558401 22769 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 01:22:58.559166 22795 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:22:58.559376 22769 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:22:58.559486 22769 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786497534195564-24578-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786497534195564-24578-0/minicluster-data/ts-1/wal
uuid: "0d7fb8d2a0fa4826a77b342b02f4cf14"
format_stamp: "Formatted at 2026-08-12 01:22:58 on dist-test-slave-t3q3"
server_key: "10b3f14a9005ca031070cff31e1760ea"
server_key_iv: "81750f16e7daa621a902beae04057a0d"
server_key_version: "encryptionkey@0"
I20260812 01:22:58.559561 22769 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786497534195564-24578-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786497534195564-24578-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786497534195564-24578-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:22:58.592497 22769 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:22:58.594820 22769 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:22:58.595023 22769 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:22:58.595283 22769 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:22:58.595618 22769 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:22:58.595675 22769 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:22:58.595723 22769 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:22:58.595755 22769 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:22:58.602377 22769 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.130:36809
I20260812 01:22:58.602427 22961 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.130:36809 every 8 connection(s)
I20260812 01:22:58.602811 22769 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786497534195564-24578-0/minicluster-data/ts-1/data/info.pb
I20260812 01:22:58.608812 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 22769
I20260812 01:22:58.609004 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786497534195564-24578-0/minicluster-data/ts-1/wal/instance
I20260812 01:22:58.609247 24578 external_mini_cluster.cc:1428] Setting key 3a99db60ba2fe0293a5ae5d9343d4ac0
Aug 12 01:22:58 dist-test-slave-t3q3 krb5kdc[22408](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497778, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.130@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
I20260812 01:22:58.613003 22967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:58.603636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:43581 (local address 127.24.0.130:52413)
0812 01:22:58.603816 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0812 01:22:58.603859 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 01:22:58.604416 (+ 557us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:22:58.604675 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:22:58.604678 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:22:58.605265 (+ 587us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:22:58.605459 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:22:58.605464 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:22:58.606283 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:22:58.606285 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:22:58.607132 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:22:58.607134 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:22:58.607533 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:22:58.607534 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:22:58.607660 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:22:58.608195 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:22:58.608205 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:22:58.609909 (+ 1704us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:22:58.611475 (+ 1566us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:22:58.611482 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:22:58.611484 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:22:58.611830 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:22:58.612107 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:22:58.612108 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:22:58.612108 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:22:58.612226 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:22:58.612511 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:22:58.612516 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:22:58.612643 (+ 127us) client_negotiation.cc:769] Sending connection context
0812 01:22:58.612720 (+ 77us) client_negotiation.cc:240] Negotiation successful
0812 01:22:58.612837 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":102,"threads_started":1}
I20260812 01:22:58.613000 22531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:58.603681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.130:52413 (local address 127.24.0.190:43581)
0812 01:22:58.603774 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0812 01:22:58.603781 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 01:22:58.603953 (+ 172us) server_negotiation.cc:407] Connection header received
0812 01:22:58.604531 (+ 578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:22:58.604532 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:22:58.604545 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:22:58.604619 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:22:58.605563 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:58.606140 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:58.607232 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:58.607450 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:58.607512 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:22:58.610063 (+ 2551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:22:58.610064 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:22:58.610065 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:22:58.610073 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:22:58.611394 (+ 1321us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:22:58.611918 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:22:58.611920 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:22:58.611920 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:22:58.611989 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:22:58.612296 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:22:58.612297 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:22:58.612298 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:22:58.612441 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:22:58.612517 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 01:22:58.612784 (+ 267us) server_negotiation.cc:299] Negotiation successful
0812 01:22:58.612885 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 01:22:58.613505 22965 heartbeater.cc:347] Connected to a master server at 127.24.0.190:43581
I20260812 01:22:58.613617 22965 heartbeater.cc:464] Registering TS with master...
I20260812 01:22:58.613811 22965 heartbeater.cc:511] Master 127.24.0.190:43581 requested a full tablet report, sending...
I20260812 01:22:58.614200 22461 ts_manager.cc:194] Registered new tserver with Master: 0d7fb8d2a0fa4826a77b342b02f4cf14 (127.24.0.130:36809)
I20260812 01:22:58.614574 22461 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.0.130@KRBTEST.COM'} at 127.24.0.130:52413
WARNING: no policy specified for kudu/127.24.0.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.0.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.0.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.131.keytab.
Entry for principal kudu/127.24.0.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.131.keytab.
I20260812 01:22:58.636539 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786497534195564-24578-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786497534195564-24578-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786497534195564-24578-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786497534195564-24578-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.0.131:0
--local_ip_for_outbound_sockets=127.24.0.131
--webserver_interface=127.24.0.131
--webserver_port=0
--tserver_master_addrs=127.24.0.190:43581
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.131.keytab
--principal=kudu/127.24.0.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:45737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:22:58.721078 22970 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:22:58.721356 22970 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:22:58.721400 22970 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:22:58.722549 22970 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:22:58.722628 22970 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:22:58.722676 22970 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.0.131
I20260812 01:22:58.724386 22970 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:45737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786497534195564-24578-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786497534195564-24578-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.131.keytab
--principal=kudu/127.24.0.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.0.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786497534195564-24578-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.0.131
--webserver_port=0
--tserver_master_addrs=127.24.0.190:43581
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22970
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.0.131
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786497534195564-24578-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:22:58.724665 22970 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:22:58.724915 22970 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:22:58.731040 22980 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:22:58.731123 22983 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:22:58.731276 22979 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:22:58.731593 22970 server_base.cc:1034] running on GCE node
I20260812 01:22:58.731837 22970 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:22:58.732219 22970 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:22:58.733438 22970 hybrid_clock.cc:648] HybridClock initialized: now 1786497778733360 us; error 41 us; skew 500 ppm
Aug 12 01:22:58 dist-test-slave-t3q3 krb5kdc[22408](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497778, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:22:58.738137 22970 init.cc:377] Logged in from keytab as kudu/127.24.0.131@KRBTEST.COM (short username kudu)
I20260812 01:22:58.738664 22970 webserver.cc:460] Webserver started at http://127.24.0.131:44177/ using document root <none> and password file <none>
I20260812 01:22:58.738919 22970 fs_manager.cc:359] Metadata directory not provided
I20260812 01:22:58.738967 22970 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:22:58.739113 22970 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:22:58.740451 22970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786497534195564-24578-0/minicluster-data/ts-2/data/instance:
uuid: "5053673fda23437fb2934156aa6bdbab"
format_stamp: "Formatted at 2026-08-12 01:22:58 on dist-test-slave-t3q3"
server_key: "52cd9a5926f088cb67220e05c373f9fc"
server_key_iv: "0b6294f6af46661bf3a9408478cf8cb1"
server_key_version: "encryptionkey@0"
I20260812 01:22:58.740837 22970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786497534195564-24578-0/minicluster-data/ts-2/wal/instance:
uuid: "5053673fda23437fb2934156aa6bdbab"
format_stamp: "Formatted at 2026-08-12 01:22:58 on dist-test-slave-t3q3"
server_key: "52cd9a5926f088cb67220e05c373f9fc"
server_key_iv: "0b6294f6af46661bf3a9408478cf8cb1"
server_key_version: "encryptionkey@0"
I20260812 01:22:58.742362 22970 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 01:22:58.743155 22994 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:22:58.743386 22970 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:22:58.743520 22970 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786497534195564-24578-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786497534195564-24578-0/minicluster-data/ts-2/wal
uuid: "5053673fda23437fb2934156aa6bdbab"
format_stamp: "Formatted at 2026-08-12 01:22:58 on dist-test-slave-t3q3"
server_key: "52cd9a5926f088cb67220e05c373f9fc"
server_key_iv: "0b6294f6af46661bf3a9408478cf8cb1"
server_key_version: "encryptionkey@0"
I20260812 01:22:58.743589 22970 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786497534195564-24578-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786497534195564-24578-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786497534195564-24578-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:22:58.760375 22970 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:22:58.762342 22970 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:22:58.762516 22970 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:22:58.762758 22970 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:22:58.763098 22970 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:22:58.763132 22970 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:22:58.763154 22970 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:22:58.763166 22970 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:22:58.768965 22970 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.131:35357
I20260812 01:22:58.769043 23171 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.131:35357 every 8 connection(s)
I20260812 01:22:58.769371 22970 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786497534195564-24578-0/minicluster-data/ts-2/data/info.pb
I20260812 01:22:58.773097 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 22970
I20260812 01:22:58.773180 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786497534195564-24578-0/minicluster-data/ts-2/wal/instance
I20260812 01:22:58.773375 24578 external_mini_cluster.cc:1428] Setting key 78e7b0730cdaa2e14d08242fe959d3d6
Aug 12 01:22:58 dist-test-slave-t3q3 krb5kdc[22408](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497778, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.131@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
I20260812 01:22:58.778820 22531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:58.770149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.131:44275 (local address 127.24.0.190:43581)
0812 01:22:58.770193 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 01:22:58.770195 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:22:58.770405 (+ 210us) server_negotiation.cc:407] Connection header received
0812 01:22:58.770918 (+ 513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:22:58.770920 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:22:58.770932 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:22:58.770999 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:22:58.771967 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:58.772513 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:58.773692 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:58.773997 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:58.774078 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:22:58.776390 (+ 2312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:22:58.776391 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:22:58.776392 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:22:58.776398 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:22:58.777662 (+ 1264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:22:58.778030 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:22:58.778031 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:22:58.778031 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:22:58.778085 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:22:58.778302 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:22:58.778302 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:22:58.778303 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:22:58.778404 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:22:58.778455 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 01:22:58.778685 (+ 230us) server_negotiation.cc:299] Negotiation successful
0812 01:22:58.778756 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 01:22:58.778977 23176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:58.770099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:43581 (local address 127.24.0.131:44275)
0812 01:22:58.770271 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0812 01:22:58.770313 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 01:22:58.770798 (+ 485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:22:58.771075 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:22:58.771078 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:22:58.771671 (+ 593us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:22:58.771867 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:22:58.771872 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:22:58.772596 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:22:58.772596 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:22:58.773592 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:22:58.773595 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:22:58.774096 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:22:58.774097 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:22:58.774258 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:22:58.774832 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:22:58.774846 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:22:58.776305 (+ 1459us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:22:58.777733 (+ 1428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:22:58.777739 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:22:58.777741 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:22:58.777957 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:22:58.778147 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:22:58.778147 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:22:58.778148 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:22:58.778245 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:22:58.778458 (+ 213us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:22:58.778462 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:22:58.778571 (+ 109us) client_negotiation.cc:769] Sending connection context
0812 01:22:58.778655 (+ 84us) client_negotiation.cc:240] Negotiation successful
0812 01:22:58.778774 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":100,"threads_started":1}
I20260812 01:22:58.779452 23174 heartbeater.cc:347] Connected to a master server at 127.24.0.190:43581
I20260812 01:22:58.779559 23174 heartbeater.cc:464] Registering TS with master...
I20260812 01:22:58.779747 23174 heartbeater.cc:511] Master 127.24.0.190:43581 requested a full tablet report, sending...
I20260812 01:22:58.780185 22461 ts_manager.cc:194] Registered new tserver with Master: 5053673fda23437fb2934156aa6bdbab (127.24.0.131:35357)
I20260812 01:22:58.780560 22461 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.0.131@KRBTEST.COM'} at 127.24.0.131:44275
I20260812 01:22:58.785355 24578 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 01:22:58.792667 22531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:58.786756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50186 (local address 127.24.0.190:43581)
0812 01:22:58.786835 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0812 01:22:58.786844 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 01:22:58.786868 (+ 24us) server_negotiation.cc:407] Connection header received
0812 01:22:58.786919 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:22:58.786920 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:22:58.786929 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:22:58.786970 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:22:58.787510 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:58.787981 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:58.788712 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:58.788888 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:58.788941 (+ 53us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:22:58.790206 (+ 1265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:22:58.790207 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:22:58.790208 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:22:58.790214 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:22:58.791400 (+ 1186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:22:58.791777 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:22:58.791778 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:22:58.791778 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:22:58.791830 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:22:58.792185 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:22:58.792186 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:22:58.792187 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:22:58.792321 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:22:58.792408 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 01:22:58.792548 (+ 140us) server_negotiation.cc:299] Negotiation successful
0812 01:22:58.792605 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 01:22:58.794296 22461 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50186:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 01:22:58.795826 22461 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 01:22:58.802800 23190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:58.798444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39300 (local address 127.24.0.129:38905)
0812 01:22:58.798706 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0812 01:22:58.798712 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 01:22:58.798725 (+ 13us) server_negotiation.cc:407] Connection header received
0812 01:22:58.798780 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:22:58.798782 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:22:58.798797 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:22:58.798874 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:22:58.799118 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:58.800237 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:58.801669 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:58.802409 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:58.802489 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:22:58.802551 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 01:22:58.802698 (+ 147us) server_negotiation.cc:299] Negotiation successful
0812 01:22:58.802735 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":228,"thread_start_us":125,"threads_started":1}
I20260812 01:22:58.802811 23185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:58.798212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.129:38905 (local address 127.0.0.1:39300)
0812 01:22:58.798568 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0812 01:22:58.798621 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 01:22:58.798689 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:22:58.798904 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:22:58.798908 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:22:58.798910 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:22:58.798993 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:22:58.798996 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:22:58.800327 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:22:58.800328 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:22:58.801584 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:22:58.801586 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:22:58.802492 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:22:58.802493 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:22:58.802579 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:22:58.802596 (+ 17us) client_negotiation.cc:769] Sending connection context
0812 01:22:58.802684 (+ 88us) client_negotiation.cc:240] Negotiation successful
0812 01:22:58.802734 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":58,"threads_started":1}
I20260812 01:22:58.803273 23186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:58.798417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.131:35357 (local address 127.0.0.1:46942)
0812 01:22:58.798958 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0812 01:22:58.798966 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 01:22:58.799000 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:22:58.799460 (+ 460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:22:58.799461 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:22:58.799463 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:22:58.799512 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:22:58.799513 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:22:58.800693 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:22:58.800694 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:22:58.801598 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:22:58.801601 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:22:58.803018 (+ 1417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:22:58.803019 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:22:58.803097 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:22:58.803102 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 01:22:58.803159 (+ 57us) client_negotiation.cc:240] Negotiation successful
0812 01:22:58.803201 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":524,"thread_start_us":87,"threads_started":1}
I20260812 01:22:58.803428 23187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:58.798287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.130:36809 (local address 127.0.0.1:51974)
0812 01:22:58.798590 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0812 01:22:58.798625 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0812 01:22:58.798691 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:22:58.800162 (+ 1471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:22:58.800163 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:22:58.800165 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:22:58.800200 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:22:58.800201 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:22:58.801521 (+ 1320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:22:58.801522 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:22:58.802300 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:22:58.802302 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:22:58.803185 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:22:58.803186 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:22:58.803253 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:22:58.803258 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 01:22:58.803322 (+ 64us) client_negotiation.cc:240] Negotiation successful
0812 01:22:58.803360 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":111,"threads_started":1}
I20260812 01:22:58.803465 23189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:58.798434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51974 (local address 127.24.0.130:36809)
0812 01:22:58.798910 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0812 01:22:58.798914 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 01:22:58.798924 (+ 10us) server_negotiation.cc:407] Connection header received
0812 01:22:58.798961 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:22:58.798963 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:22:58.798980 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:22:58.799066 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:22:58.800275 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:58.801412 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:58.802384 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:58.803115 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:58.803181 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:22:58.803239 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 01:22:58.803333 (+ 94us) server_negotiation.cc:299] Negotiation successful
0812 01:22:58.803382 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":439,"thread_start_us":78,"threads_started":1}
I20260812 01:22:58.803716 22656 tablet_service.cc:1467] Processing CreateTablet for tablet 10673481282e42a0b73d23afc6c49a4b (DEFAULT_TABLE table=test-table [id=bdd732d3f5f547b79136fc367e0225e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:22:58.804139 22656 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10673481282e42a0b73d23afc6c49a4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:22:58.804562 23191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:58.798680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46942 (local address 127.24.0.131:35357)
0812 01:22:58.799240 (+ 560us) server_negotiation.cc:206] Beginning negotiation
0812 01:22:58.799244 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 01:22:58.799254 (+ 10us) server_negotiation.cc:407] Connection header received
0812 01:22:58.799292 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:22:58.799294 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:22:58.799309 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:22:58.799381 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:22:58.799591 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:58.800589 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:58.801712 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:58.802408 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:58.802489 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:22:58.802559 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 01:22:58.804432 (+ 1873us) server_negotiation.cc:299] Negotiation successful
0812 01:22:58.804482 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":521,"thread_start_us":80,"threads_started":1}
I20260812 01:22:58.804766 22862 tablet_service.cc:1467] Processing CreateTablet for tablet 10673481282e42a0b73d23afc6c49a4b (DEFAULT_TABLE table=test-table [id=bdd732d3f5f547b79136fc367e0225e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:22:58.805117 22862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10673481282e42a0b73d23afc6c49a4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:22:58.805239 23071 tablet_service.cc:1467] Processing CreateTablet for tablet 10673481282e42a0b73d23afc6c49a4b (DEFAULT_TABLE table=test-table [id=bdd732d3f5f547b79136fc367e0225e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:22:58.805568 23071 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10673481282e42a0b73d23afc6c49a4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:22:58.807842 23195 tablet_bootstrap.cc:492] T 10673481282e42a0b73d23afc6c49a4b P 0c20a859489745fbb1f178c0c455885a: Bootstrap starting.
I20260812 01:22:58.808395 23196 tablet_bootstrap.cc:492] T 10673481282e42a0b73d23afc6c49a4b P 5053673fda23437fb2934156aa6bdbab: Bootstrap starting.
I20260812 01:22:58.808950 23195 tablet_bootstrap.cc:654] T 10673481282e42a0b73d23afc6c49a4b P 0c20a859489745fbb1f178c0c455885a: Neither blocks nor log segments found. Creating new log.
I20260812 01:22:58.809002 23198 tablet_bootstrap.cc:492] T 10673481282e42a0b73d23afc6c49a4b P 0d7fb8d2a0fa4826a77b342b02f4cf14: Bootstrap starting.
I20260812 01:22:58.809099 23196 tablet_bootstrap.cc:654] T 10673481282e42a0b73d23afc6c49a4b P 5053673fda23437fb2934156aa6bdbab: Neither blocks nor log segments found. Creating new log.
I20260812 01:22:58.809630 23195 log.cc:824] T 10673481282e42a0b73d23afc6c49a4b P 0c20a859489745fbb1f178c0c455885a: Log is configured to *not* fsync() on all Append() calls
I20260812 01:22:58.809736 23198 tablet_bootstrap.cc:654] T 10673481282e42a0b73d23afc6c49a4b P 0d7fb8d2a0fa4826a77b342b02f4cf14: Neither blocks nor log segments found. Creating new log.
I20260812 01:22:58.809731 23196 log.cc:824] T 10673481282e42a0b73d23afc6c49a4b P 5053673fda23437fb2934156aa6bdbab: Log is configured to *not* fsync() on all Append() calls
I20260812 01:22:58.810362 23198 log.cc:824] T 10673481282e42a0b73d23afc6c49a4b P 0d7fb8d2a0fa4826a77b342b02f4cf14: Log is configured to *not* fsync() on all Append() calls
I20260812 01:22:58.810494 23196 tablet_bootstrap.cc:492] T 10673481282e42a0b73d23afc6c49a4b P 5053673fda23437fb2934156aa6bdbab: No bootstrap required, opened a new log
I20260812 01:22:58.810570 23196 ts_tablet_manager.cc:1397] T 10673481282e42a0b73d23afc6c49a4b P 5053673fda23437fb2934156aa6bdbab: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 01:22:58.810554 23195 tablet_bootstrap.cc:492] T 10673481282e42a0b73d23afc6c49a4b P 0c20a859489745fbb1f178c0c455885a: No bootstrap required, opened a new log
I20260812 01:22:58.810652 23195 ts_tablet_manager.cc:1397] T 10673481282e42a0b73d23afc6c49a4b P 0c20a859489745fbb1f178c0c455885a: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 01:22:58.812335 23196 raft_consensus.cc:348] T 10673481282e42a0b73d23afc6c49a4b P 5053673fda23437fb2934156aa6bdbab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c20a859489745fbb1f178c0c455885a" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 38905 } } peers { permanent_uuid: "5053673fda23437fb2934156aa6bdbab" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 35357 } } peers { permanent_uuid: "0d7fb8d2a0fa4826a77b342b02f4cf14" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 36809 } }
I20260812 01:22:58.812551 23196 raft_consensus.cc:374] T 10673481282e42a0b73d23afc6c49a4b P 5053673fda23437fb2934156aa6bdbab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:22:58.812593 23196 raft_consensus.cc:729] T 10673481282e42a0b73d23afc6c49a4b P 5053673fda23437fb2934156aa6bdbab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5053673fda23437fb2934156aa6bdbab, State: Initialized, Role: FOLLOWER
I20260812 01:22:58.812497 23195 raft_consensus.cc:348] T 10673481282e42a0b73d23afc6c49a4b P 0c20a859489745fbb1f178c0c455885a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c20a859489745fbb1f178c0c455885a" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 38905 } } peers { permanent_uuid: "5053673fda23437fb2934156aa6bdbab" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 35357 } } peers { permanent_uuid: "0d7fb8d2a0fa4826a77b342b02f4cf14" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 36809 } }
I20260812 01:22:58.812635 23195 raft_consensus.cc:374] T 10673481282e42a0b73d23afc6c49a4b P 0c20a859489745fbb1f178c0c455885a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:22:58.812657 23195 raft_consensus.cc:729] T 10673481282e42a0b73d23afc6c49a4b P 0c20a859489745fbb1f178c0c455885a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c20a859489745fbb1f178c0c455885a, State: Initialized, Role: FOLLOWER
I20260812 01:22:58.812749 23198 tablet_bootstrap.cc:492] T 10673481282e42a0b73d23afc6c49a4b P 0d7fb8d2a0fa4826a77b342b02f4cf14: No bootstrap required, opened a new log
I20260812 01:22:58.812717 23196 consensus_queue.cc:260] T 10673481282e42a0b73d23afc6c49a4b P 5053673fda23437fb2934156aa6bdbab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c20a859489745fbb1f178c0c455885a" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 38905 } } peers { permanent_uuid: "5053673fda23437fb2934156aa6bdbab" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 35357 } } peers { permanent_uuid: "0d7fb8d2a0fa4826a77b342b02f4cf14" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 36809 } }
I20260812 01:22:58.812806 23198 ts_tablet_manager.cc:1397] T 10673481282e42a0b73d23afc6c49a4b P 0d7fb8d2a0fa4826a77b342b02f4cf14: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260812 01:22:58.812777 23195 consensus_queue.cc:260] T 10673481282e42a0b73d23afc6c49a4b P 0c20a859489745fbb1f178c0c455885a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c20a859489745fbb1f178c0c455885a" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 38905 } } peers { permanent_uuid: "5053673fda23437fb2934156aa6bdbab" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 35357 } } peers { permanent_uuid: "0d7fb8d2a0fa4826a77b342b02f4cf14" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 36809 } }
I20260812 01:22:58.813050 23196 ts_tablet_manager.cc:1428] T 10673481282e42a0b73d23afc6c49a4b P 5053673fda23437fb2934156aa6bdbab: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 01:22:58.813077 23195 ts_tablet_manager.cc:1428] T 10673481282e42a0b73d23afc6c49a4b P 0c20a859489745fbb1f178c0c455885a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 01:22:58.813097 23174 heartbeater.cc:503] Master 127.24.0.190:43581 was elected leader, sending a full tablet report...
I20260812 01:22:58.813206 22758 heartbeater.cc:503] Master 127.24.0.190:43581 was elected leader, sending a full tablet report...
I20260812 01:22:58.814833 23198 raft_consensus.cc:348] T 10673481282e42a0b73d23afc6c49a4b P 0d7fb8d2a0fa4826a77b342b02f4cf14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c20a859489745fbb1f178c0c455885a" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 38905 } } peers { permanent_uuid: "5053673fda23437fb2934156aa6bdbab" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 35357 } } peers { permanent_uuid: "0d7fb8d2a0fa4826a77b342b02f4cf14" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 36809 } }
I20260812 01:22:58.814958 23198 raft_consensus.cc:374] T 10673481282e42a0b73d23afc6c49a4b P 0d7fb8d2a0fa4826a77b342b02f4cf14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:22:58.814980 23198 raft_consensus.cc:729] T 10673481282e42a0b73d23afc6c49a4b P 0d7fb8d2a0fa4826a77b342b02f4cf14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d7fb8d2a0fa4826a77b342b02f4cf14, State: Initialized, Role: FOLLOWER
I20260812 01:22:58.815065 23198 consensus_queue.cc:260] T 10673481282e42a0b73d23afc6c49a4b P 0d7fb8d2a0fa4826a77b342b02f4cf14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c20a859489745fbb1f178c0c455885a" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 38905 } } peers { permanent_uuid: "5053673fda23437fb2934156aa6bdbab" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 35357 } } peers { permanent_uuid: "0d7fb8d2a0fa4826a77b342b02f4cf14" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 36809 } }
I20260812 01:22:58.815268 23198 ts_tablet_manager.cc:1428] T 10673481282e42a0b73d23afc6c49a4b P 0d7fb8d2a0fa4826a77b342b02f4cf14: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 01:22:58.815341 22965 heartbeater.cc:503] Master 127.24.0.190:43581 was elected leader, sending a full tablet report...
W20260812 01:22:58.853416 22966 tablet.cc:2369] T 10673481282e42a0b73d23afc6c49a4b P 0d7fb8d2a0fa4826a77b342b02f4cf14: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 01:22:58.927045 22759 tablet.cc:2369] T 10673481282e42a0b73d23afc6c49a4b P 0c20a859489745fbb1f178c0c455885a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 01:22:59.019976 23175 tablet.cc:2369] T 10673481282e42a0b73d23afc6c49a4b P 5053673fda23437fb2934156aa6bdbab: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 01:22:59.162549 23204 raft_consensus.cc:482] T 10673481282e42a0b73d23afc6c49a4b P 0c20a859489745fbb1f178c0c455885a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 01:22:59.162686 23204 raft_consensus.cc:504] T 10673481282e42a0b73d23afc6c49a4b P 0c20a859489745fbb1f178c0c455885a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c20a859489745fbb1f178c0c455885a" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 38905 } } peers { permanent_uuid: "5053673fda23437fb2934156aa6bdbab" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 35357 } } peers { permanent_uuid: "0d7fb8d2a0fa4826a77b342b02f4cf14" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 36809 } }
I20260812 01:22:59.163144 23204 leader_election.cc:288] T 10673481282e42a0b73d23afc6c49a4b P 0c20a859489745fbb1f178c0c455885a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5053673fda23437fb2934156aa6bdbab (127.24.0.131:35357), 0d7fb8d2a0fa4826a77b342b02f4cf14 (127.24.0.130:36809)
I20260812 01:22:59.166198 23189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:59.163453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.129:52857 (local address 127.24.0.130:36809)
0812 01:22:59.163507 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0812 01:22:59.163509 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:22:59.163634 (+ 125us) server_negotiation.cc:407] Connection header received
0812 01:22:59.163715 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:22:59.163716 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:22:59.163728 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:22:59.163801 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:22:59.163998 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:59.164511 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:59.165467 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:59.165818 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:59.165903 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:22:59.165912 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 01:22:59.166063 (+ 151us) server_negotiation.cc:299] Negotiation successful
0812 01:22:59.166125 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 01:22:59.166222 23208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:59.163373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.130:36809 (local address 127.24.0.129:52857)
0812 01:22:59.163527 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0812 01:22:59.163545 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 01:22:59.163652 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:22:59.163822 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:22:59.163823 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:22:59.163825 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:22:59.163920 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:22:59.163921 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:22:59.164609 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:22:59.164610 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:22:59.165363 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:22:59.165365 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:22:59.165918 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:22:59.165919 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:22:59.165990 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:22:59.165994 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 01:22:59.166058 (+ 64us) client_negotiation.cc:240] Negotiation successful
0812 01:22:59.166100 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"mutex_wait_us":1,"thread_start_us":85,"threads_started":1}
I20260812 01:22:59.166224 23191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:59.163536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.129:45631 (local address 127.24.0.131:35357)
0812 01:22:59.163587 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 01:22:59.163589 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:22:59.163601 (+ 12us) server_negotiation.cc:407] Connection header received
0812 01:22:59.163660 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:22:59.163661 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:22:59.163672 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:22:59.163753 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:22:59.163944 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:59.164473 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:59.165466 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:59.165853 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:59.165944 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:22:59.165953 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 01:22:59.166118 (+ 165us) server_negotiation.cc:299] Negotiation successful
0812 01:22:59.166157 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260812 01:22:59.166244 23211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:59.163326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.131:35357 (local address 127.24.0.129:45631)
0812 01:22:59.163516 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0812 01:22:59.163527 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 01:22:59.163593 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:22:59.163763 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:22:59.163764 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:22:59.163766 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:22:59.163848 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:22:59.163850 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:22:59.164569 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:22:59.164570 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:22:59.165364 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:22:59.165366 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:22:59.165945 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:22:59.165947 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:22:59.166027 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:22:59.166031 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 01:22:59.166109 (+ 78us) client_negotiation.cc:240] Negotiation successful
0812 01:22:59.166152 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":100,"threads_started":1}
I20260812 01:22:59.166584 23102 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10673481282e42a0b73d23afc6c49a4b" candidate_uuid: "0c20a859489745fbb1f178c0c455885a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5053673fda23437fb2934156aa6bdbab" is_pre_election: true
I20260812 01:22:59.166790 23102 raft_consensus.cc:2454] T 10673481282e42a0b73d23afc6c49a4b P 5053673fda23437fb2934156aa6bdbab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c20a859489745fbb1f178c0c455885a in term 0.
I20260812 01:22:59.166749 22894 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10673481282e42a0b73d23afc6c49a4b" candidate_uuid: "0c20a859489745fbb1f178c0c455885a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d7fb8d2a0fa4826a77b342b02f4cf14" is_pre_election: true
I20260812 01:22:59.166886 22894 raft_consensus.cc:2454] T 10673481282e42a0b73d23afc6c49a4b P 0d7fb8d2a0fa4826a77b342b02f4cf14 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c20a859489745fbb1f178c0c455885a in term 0.
I20260812 01:22:59.167069 22585 leader_election.cc:302] T 10673481282e42a0b73d23afc6c49a4b P 0c20a859489745fbb1f178c0c455885a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c20a859489745fbb1f178c0c455885a, 5053673fda23437fb2934156aa6bdbab; no voters:
I20260812 01:22:59.167210 23204 raft_consensus.cc:2781] T 10673481282e42a0b73d23afc6c49a4b P 0c20a859489745fbb1f178c0c455885a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 01:22:59.167280 23204 raft_consensus.cc:482] T 10673481282e42a0b73d23afc6c49a4b P 0c20a859489745fbb1f178c0c455885a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 01:22:59.167299 23204 raft_consensus.cc:3037] T 10673481282e42a0b73d23afc6c49a4b P 0c20a859489745fbb1f178c0c455885a [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:22:59.168148 23204 raft_consensus.cc:504] T 10673481282e42a0b73d23afc6c49a4b P 0c20a859489745fbb1f178c0c455885a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c20a859489745fbb1f178c0c455885a" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 38905 } } peers { permanent_uuid: "5053673fda23437fb2934156aa6bdbab" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 35357 } } peers { permanent_uuid: "0d7fb8d2a0fa4826a77b342b02f4cf14" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 36809 } }
I20260812 01:22:59.168303 23204 leader_election.cc:288] T 10673481282e42a0b73d23afc6c49a4b P 0c20a859489745fbb1f178c0c455885a [CANDIDATE]: Term 1 election: Requested vote from peers 5053673fda23437fb2934156aa6bdbab (127.24.0.131:35357), 0d7fb8d2a0fa4826a77b342b02f4cf14 (127.24.0.130:36809)
I20260812 01:22:59.168511 23102 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10673481282e42a0b73d23afc6c49a4b" candidate_uuid: "0c20a859489745fbb1f178c0c455885a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5053673fda23437fb2934156aa6bdbab"
I20260812 01:22:59.168495 22894 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10673481282e42a0b73d23afc6c49a4b" candidate_uuid: "0c20a859489745fbb1f178c0c455885a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d7fb8d2a0fa4826a77b342b02f4cf14"
I20260812 01:22:59.168604 22894 raft_consensus.cc:3037] T 10673481282e42a0b73d23afc6c49a4b P 0d7fb8d2a0fa4826a77b342b02f4cf14 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:22:59.168634 23102 raft_consensus.cc:3037] T 10673481282e42a0b73d23afc6c49a4b P 5053673fda23437fb2934156aa6bdbab [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:22:59.169344 23102 raft_consensus.cc:2454] T 10673481282e42a0b73d23afc6c49a4b P 5053673fda23437fb2934156aa6bdbab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c20a859489745fbb1f178c0c455885a in term 1.
I20260812 01:22:59.169361 22894 raft_consensus.cc:2454] T 10673481282e42a0b73d23afc6c49a4b P 0d7fb8d2a0fa4826a77b342b02f4cf14 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c20a859489745fbb1f178c0c455885a in term 1.
I20260812 01:22:59.169523 22585 leader_election.cc:302] T 10673481282e42a0b73d23afc6c49a4b P 0c20a859489745fbb1f178c0c455885a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c20a859489745fbb1f178c0c455885a, 5053673fda23437fb2934156aa6bdbab; no voters:
I20260812 01:22:59.169620 23204 raft_consensus.cc:2781] T 10673481282e42a0b73d23afc6c49a4b P 0c20a859489745fbb1f178c0c455885a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:22:59.169780 23204 raft_consensus.cc:686] T 10673481282e42a0b73d23afc6c49a4b P 0c20a859489745fbb1f178c0c455885a [term 1 LEADER]: Becoming Leader. State: Replica: 0c20a859489745fbb1f178c0c455885a, State: Running, Role: LEADER
I20260812 01:22:59.169893 23204 consensus_queue.cc:237] T 10673481282e42a0b73d23afc6c49a4b P 0c20a859489745fbb1f178c0c455885a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c20a859489745fbb1f178c0c455885a" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 38905 } } peers { permanent_uuid: "5053673fda23437fb2934156aa6bdbab" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 35357 } } peers { permanent_uuid: "0d7fb8d2a0fa4826a77b342b02f4cf14" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 36809 } }
I20260812 01:22:59.170773 22460 catalog_manager.cc:5495] T 10673481282e42a0b73d23afc6c49a4b P 0c20a859489745fbb1f178c0c455885a reported cstate change: term changed from 0 to 1, leader changed from <none> to 0c20a859489745fbb1f178c0c455885a (127.24.0.129). New cstate: current_term: 1 leader_uuid: "0c20a859489745fbb1f178c0c455885a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c20a859489745fbb1f178c0c455885a" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 38905 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5053673fda23437fb2934156aa6bdbab" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 35357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d7fb8d2a0fa4826a77b342b02f4cf14" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 36809 } health_report { overall_health: UNKNOWN } } }
I20260812 01:22:59.245674 23190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:59.243415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39304 (local address 127.24.0.129:38905)
0812 01:22:59.243480 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0812 01:22:59.243482 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:22:59.243493 (+ 11us) server_negotiation.cc:407] Connection header received
0812 01:22:59.243535 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:22:59.243536 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:22:59.243557 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:22:59.243620 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:22:59.243755 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:59.244248 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:59.244948 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:59.245131 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:59.245190 (+ 59us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:22:59.245330 (+ 140us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:22:59.245414 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:22:59.245482 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 01:22:59.245531 (+ 49us) server_negotiation.cc:299] Negotiation successful
0812 01:22:59.245541 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 01:22:59.247570 22894 raft_consensus.cc:1261] T 10673481282e42a0b73d23afc6c49a4b P 0d7fb8d2a0fa4826a77b342b02f4cf14 [term 1 FOLLOWER]: Refusing update from remote peer 0c20a859489745fbb1f178c0c455885a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:22:59.247560 23102 raft_consensus.cc:1261] T 10673481282e42a0b73d23afc6c49a4b P 5053673fda23437fb2934156aa6bdbab [term 1 FOLLOWER]: Refusing update from remote peer 0c20a859489745fbb1f178c0c455885a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:22:59.247951 23212 consensus_queue.cc:1035] T 10673481282e42a0b73d23afc6c49a4b P 0c20a859489745fbb1f178c0c455885a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d7fb8d2a0fa4826a77b342b02f4cf14" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 36809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:22:59.248131 23204 consensus_queue.cc:1035] T 10673481282e42a0b73d23afc6c49a4b P 0c20a859489745fbb1f178c0c455885a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5053673fda23437fb2934156aa6bdbab" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 35357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:22:59.250452 23216 mvcc.cc:205] Tried to move back new op lower bound from 7317494903794913280 to 7317494903480573952. Current Snapshot: MvccSnapshot[applied={T|T < 7317494903794913280}]
I20260812 01:22:59.250643 23217 mvcc.cc:205] Tried to move back new op lower bound from 7317494903794913280 to 7317494903480573952. Current Snapshot: MvccSnapshot[applied={T|T < 7317494903794913280}]
I20260812 01:22:59.251655 23218 mvcc.cc:205] Tried to move back new op lower bound from 7317494903794913280 to 7317494903480573952. Current Snapshot: MvccSnapshot[applied={T|T < 7317494903794913280 or (T in {7317494903794913280})}]
I20260812 01:22:59.252897 22459 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50186:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:22:59.253001 22459 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50186:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:22:59.253757 22459 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6fd156381b734def8eeac63b70845f2f: Sending DeleteTablet for 3 replicas of tablet 10673481282e42a0b73d23afc6c49a4b
I20260812 01:22:59.254081 22656 tablet_service.cc:1514] Processing DeleteTablet for tablet 10673481282e42a0b73d23afc6c49a4b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:22:59 UTC) from {username='kudu', principal='kudu/127.24.0.190@KRBTEST.COM'} at 127.0.0.1:39300
I20260812 01:22:59.254108 22862 tablet_service.cc:1514] Processing DeleteTablet for tablet 10673481282e42a0b73d23afc6c49a4b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:22:59 UTC) from {username='kudu', principal='kudu/127.24.0.190@KRBTEST.COM'} at 127.0.0.1:51974
I20260812 01:22:59.254290 23227 tablet_replica.cc:330] T 10673481282e42a0b73d23afc6c49a4b P 0c20a859489745fbb1f178c0c455885a: stopping tablet replica
I20260812 01:22:59.254312 23071 tablet_service.cc:1514] Processing DeleteTablet for tablet 10673481282e42a0b73d23afc6c49a4b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:22:59 UTC) from {username='kudu', principal='kudu/127.24.0.190@KRBTEST.COM'} at 127.0.0.1:46942
I20260812 01:22:59.254413 23227 raft_consensus.cc:2229] T 10673481282e42a0b73d23afc6c49a4b P 0c20a859489745fbb1f178c0c455885a [term 1 LEADER]: Raft consensus shutting down.
I20260812 01:22:59.254514 23227 raft_consensus.cc:2258] T 10673481282e42a0b73d23afc6c49a4b P 0c20a859489745fbb1f178c0c455885a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:22:59.254549 23231 tablet_replica.cc:330] T 10673481282e42a0b73d23afc6c49a4b P 5053673fda23437fb2934156aa6bdbab: stopping tablet replica
I20260812 01:22:59.254688 23231 raft_consensus.cc:2229] T 10673481282e42a0b73d23afc6c49a4b P 5053673fda23437fb2934156aa6bdbab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 01:22:59.254787 23230 tablet_replica.cc:330] T 10673481282e42a0b73d23afc6c49a4b P 0d7fb8d2a0fa4826a77b342b02f4cf14: stopping tablet replica
I20260812 01:22:59.254838 23231 raft_consensus.cc:2258] T 10673481282e42a0b73d23afc6c49a4b P 5053673fda23437fb2934156aa6bdbab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:22:59.254866 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 22558
I20260812 01:22:59.254925 23230 raft_consensus.cc:2229] T 10673481282e42a0b73d23afc6c49a4b P 0d7fb8d2a0fa4826a77b342b02f4cf14 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 01:22:59.255053 23230 raft_consensus.cc:2258] T 10673481282e42a0b73d23afc6c49a4b P 0d7fb8d2a0fa4826a77b342b02f4cf14 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:22:59.274153 23230 ts_tablet_manager.cc:1905] T 10673481282e42a0b73d23afc6c49a4b P 0d7fb8d2a0fa4826a77b342b02f4cf14: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:22:59.275808 23231 ts_tablet_manager.cc:1905] T 10673481282e42a0b73d23afc6c49a4b P 5053673fda23437fb2934156aa6bdbab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:22:59.276427 23230 ts_tablet_manager.cc:1918] T 10673481282e42a0b73d23afc6c49a4b P 0d7fb8d2a0fa4826a77b342b02f4cf14: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 01:22:59.276518 23230 log.cc:1196] T 10673481282e42a0b73d23afc6c49a4b P 0d7fb8d2a0fa4826a77b342b02f4cf14: Deleting WAL directory at /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786497534195564-24578-0/minicluster-data/ts-1/wal/wals/10673481282e42a0b73d23afc6c49a4b
I20260812 01:22:59.276788 23230 ts_tablet_manager.cc:1939] T 10673481282e42a0b73d23afc6c49a4b P 0d7fb8d2a0fa4826a77b342b02f4cf14: Deleting consensus metadata
I20260812 01:22:59.277369 23231 ts_tablet_manager.cc:1918] T 10673481282e42a0b73d23afc6c49a4b P 5053673fda23437fb2934156aa6bdbab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 01:22:59.277469 23231 log.cc:1196] T 10673481282e42a0b73d23afc6c49a4b P 5053673fda23437fb2934156aa6bdbab: Deleting WAL directory at /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786497534195564-24578-0/minicluster-data/ts-2/wal/wals/10673481282e42a0b73d23afc6c49a4b
I20260812 01:22:59.277778 23231 ts_tablet_manager.cc:1939] T 10673481282e42a0b73d23afc6c49a4b P 5053673fda23437fb2934156aa6bdbab: Deleting consensus metadata
I20260812 01:22:59.277761 22442 catalog_manager.cc:4841] TS 0d7fb8d2a0fa4826a77b342b02f4cf14 (127.24.0.130:36809): tablet 10673481282e42a0b73d23afc6c49a4b (table test-table [id=bdd732d3f5f547b79136fc367e0225e1]) successfully deleted
I20260812 01:22:59.278234 22445 catalog_manager.cc:4841] TS 5053673fda23437fb2934156aa6bdbab (127.24.0.131:35357): tablet 10673481282e42a0b73d23afc6c49a4b (table test-table [id=bdd732d3f5f547b79136fc367e0225e1]) successfully deleted
W20260812 01:22:59.280427 22445 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.0.129:38905 (error 108)
I20260812 01:22:59.280660 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 22769
I20260812 01:22:59.288939 23187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:59.284298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.129:38905 (local address 127.0.0.1:39318)
0812 01:22:59.288850 (+ 4552us) negotiation.cc:107] Waiting for socket to connect
0812 01:22:59.288905 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.129:38905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4523}
W20260812 01:22:59.292347 22445 catalog_manager.cc:4584] TS 0c20a859489745fbb1f178c0c455885a (127.24.0.129:38905): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 10673481282e42a0b73d23afc6c49a4b: Network error: Client connection negotiation failed: client connection to 127.24.0.129:38905: connect: Connection refused (error 111)
I20260812 01:22:59.293259 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 22970
I20260812 01:22:59.301318 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 22421
2026-08-12T01:22:59Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2609 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-12T01:22:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:22:59Z Disabled control of system clock
I20260812 01:22:59.337941 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786497534195564-24578-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786497534195564-24578-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786497534195564-24578-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.190:34415
--webserver_interface=127.24.0.190
--webserver_port=0
--builtin_ntp_servers=127.24.0.148:34891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.0.190:34415
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 01:22:59.419692 23236 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:22:59.419921 23236 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:22:59.419941 23236 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:22:59.421146 23236 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:22:59.421195 23236 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:22:59.421208 23236 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:22:59.421221 23236 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:22:59.421277 23236 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:22:59.422677 23236 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:34891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786497534195564-24578-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786497534195564-24578-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.0.190:34415
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.190:34415
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.0.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23236
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786497534195564-24578-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:22:59.422943 23236 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:22:59.423254 23236 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:22:59.428308 23242 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:22:59.428308 23243 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:22:59.428385 23245 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:22:59.428490 23236 server_base.cc:1034] running on GCE node
I20260812 01:22:59.428920 23236 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:22:59.429250 23236 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:22:59.430429 23236 hybrid_clock.cc:648] HybridClock initialized: now 1786497779430412 us; error 53 us; skew 500 ppm
I20260812 01:22:59.431845 23236 webserver.cc:460] Webserver started at http://127.24.0.190:35117/ using document root <none> and password file <none>
I20260812 01:22:59.432188 23236 fs_manager.cc:359] Metadata directory not provided
I20260812 01:22:59.432263 23236 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:22:59.432410 23236 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:22:59.433535 23236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786497534195564-24578-0/minicluster-data/master-0/data/instance:
uuid: "fc10eb8737fd4f7188eb24e60b7d2083"
format_stamp: "Formatted at 2026-08-12 01:22:59 on dist-test-slave-t3q3"
server_key: "e336cc74c6761c4e864a2e962d700a64"
server_key_iv: "b87f817aa8601eb317e68e73bafa07e9"
server_key_version: "encryptionkey@0"
I20260812 01:22:59.433912 23236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786497534195564-24578-0/minicluster-data/master-0/wal/instance:
uuid: "fc10eb8737fd4f7188eb24e60b7d2083"
format_stamp: "Formatted at 2026-08-12 01:22:59 on dist-test-slave-t3q3"
server_key: "e336cc74c6761c4e864a2e962d700a64"
server_key_iv: "b87f817aa8601eb317e68e73bafa07e9"
server_key_version: "encryptionkey@0"
I20260812 01:22:59.435245 23236 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 01:22:59.436039 23256 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:22:59.436265 23236 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 01:22:59.436367 23236 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786497534195564-24578-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786497534195564-24578-0/minicluster-data/master-0/wal
uuid: "fc10eb8737fd4f7188eb24e60b7d2083"
format_stamp: "Formatted at 2026-08-12 01:22:59 on dist-test-slave-t3q3"
server_key: "e336cc74c6761c4e864a2e962d700a64"
server_key_iv: "b87f817aa8601eb317e68e73bafa07e9"
server_key_version: "encryptionkey@0"
I20260812 01:22:59.436455 23236 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786497534195564-24578-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786497534195564-24578-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786497534195564-24578-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:22:59.464362 23236 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:22:59.464704 23236 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:22:59.464871 23236 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:22:59.469070 23236 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.190:34415
I20260812 01:22:59.469107 23336 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.190:34415 every 8 connection(s)
I20260812 01:22:59.469455 23236 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
I20260812 01:22:59.470469 23337 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:22:59.472680 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 23236
I20260812 01:22:59.472800 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786497534195564-24578-0/minicluster-data/master-0/wal/instance
I20260812 01:22:59.473033 24578 external_mini_cluster.cc:1428] Setting key c91ce65eec5c3664ac6004bc075a204e
I20260812 01:22:59.473127 23337 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc10eb8737fd4f7188eb24e60b7d2083: Bootstrap starting.
I20260812 01:22:59.473883 23337 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fc10eb8737fd4f7188eb24e60b7d2083: Neither blocks nor log segments found. Creating new log.
I20260812 01:22:59.474205 23337 log.cc:824] T 00000000000000000000000000000000 P fc10eb8737fd4f7188eb24e60b7d2083: Log is configured to *not* fsync() on all Append() calls
I20260812 01:22:59.475031 23337 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc10eb8737fd4f7188eb24e60b7d2083: No bootstrap required, opened a new log
I20260812 01:22:59.477244 23337 raft_consensus.cc:348] T 00000000000000000000000000000000 P fc10eb8737fd4f7188eb24e60b7d2083 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc10eb8737fd4f7188eb24e60b7d2083" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 34415 } }
I20260812 01:22:59.477380 23337 raft_consensus.cc:374] T 00000000000000000000000000000000 P fc10eb8737fd4f7188eb24e60b7d2083 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:22:59.477401 23337 raft_consensus.cc:729] T 00000000000000000000000000000000 P fc10eb8737fd4f7188eb24e60b7d2083 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc10eb8737fd4f7188eb24e60b7d2083, State: Initialized, Role: FOLLOWER
I20260812 01:22:59.477531 23337 consensus_queue.cc:260] T 00000000000000000000000000000000 P fc10eb8737fd4f7188eb24e60b7d2083 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc10eb8737fd4f7188eb24e60b7d2083" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 34415 } }
I20260812 01:22:59.477626 23337 raft_consensus.cc:388] T 00000000000000000000000000000000 P fc10eb8737fd4f7188eb24e60b7d2083 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:22:59.477669 23337 raft_consensus.cc:482] T 00000000000000000000000000000000 P fc10eb8737fd4f7188eb24e60b7d2083 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:22:59.477710 23337 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fc10eb8737fd4f7188eb24e60b7d2083 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:22:59.478406 23337 raft_consensus.cc:504] T 00000000000000000000000000000000 P fc10eb8737fd4f7188eb24e60b7d2083 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc10eb8737fd4f7188eb24e60b7d2083" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 34415 } }
I20260812 01:22:59.478502 23337 leader_election.cc:302] T 00000000000000000000000000000000 P fc10eb8737fd4f7188eb24e60b7d2083 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fc10eb8737fd4f7188eb24e60b7d2083; no voters:
I20260812 01:22:59.478700 23337 leader_election.cc:288] T 00000000000000000000000000000000 P fc10eb8737fd4f7188eb24e60b7d2083 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 01:22:59.478796 23345 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fc10eb8737fd4f7188eb24e60b7d2083 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:22:59.478972 23345 raft_consensus.cc:686] T 00000000000000000000000000000000 P fc10eb8737fd4f7188eb24e60b7d2083 [term 1 LEADER]: Becoming Leader. State: Replica: fc10eb8737fd4f7188eb24e60b7d2083, State: Running, Role: LEADER
I20260812 01:22:59.479008 23337 sys_catalog.cc:564] T 00000000000000000000000000000000 P fc10eb8737fd4f7188eb24e60b7d2083 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:22:59.479138 23345 consensus_queue.cc:237] T 00000000000000000000000000000000 P fc10eb8737fd4f7188eb24e60b7d2083 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc10eb8737fd4f7188eb24e60b7d2083" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 34415 } }
I20260812 01:22:59.479219 23342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:59.473614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53328 (local address 127.24.0.190:34415)
0812 01:22:59.473833 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0812 01:22:59.473840 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 01:22:59.473865 (+ 25us) server_negotiation.cc:407] Connection header received
0812 01:22:59.474420 (+ 555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:22:59.474431 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:22:59.474519 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:22:59.474675 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:22:59.474934 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:59.475869 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:59.477052 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:59.477552 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:59.477706 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:22:59.478341 (+ 635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:22:59.478344 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:22:59.478345 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:22:59.478551 (+ 206us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:22:59.478600 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:22:59.478616 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:22:59.478704 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0812 01:22:59.478791 (+ 87us) server_negotiation.cc:299] Negotiation successful
0812 01:22:59.478879 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":98,"threads_started":1}
I20260812 01:22:59.479507 23345 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc10eb8737fd4f7188eb24e60b7d2083 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fc10eb8737fd4f7188eb24e60b7d2083. Latest consensus state: current_term: 1 leader_uuid: "fc10eb8737fd4f7188eb24e60b7d2083" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc10eb8737fd4f7188eb24e60b7d2083" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 34415 } } }
I20260812 01:22:59.479598 23345 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc10eb8737fd4f7188eb24e60b7d2083 [sys.catalog]: This master's current role is: LEADER
I20260812 01:22:59.479517 23347 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc10eb8737fd4f7188eb24e60b7d2083 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fc10eb8737fd4f7188eb24e60b7d2083" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc10eb8737fd4f7188eb24e60b7d2083" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 34415 } } }
I20260812 01:22:59.479645 23347 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc10eb8737fd4f7188eb24e60b7d2083 [sys.catalog]: This master's current role is: LEADER
I20260812 01:22:59.479811 23354 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:22:59.480319 23354 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:22:59.485194 23354 catalog_manager.cc:1347] Generated new cluster ID: bc3d7405c82847319c8c0c7083bd1cf8
I20260812 01:22:59.485260 23354 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 01:22:59.493920 23354 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:22:59.494462 23354 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:22:59.500346 23354 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fc10eb8737fd4f7188eb24e60b7d2083: Generated new TSK 0
I20260812 01:22:59.500541 23354 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 01:22:59.509253 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786497534195564-24578-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786497534195564-24578-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786497534195564-24578-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786497534195564-24578-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.0.129:0
--local_ip_for_outbound_sockets=127.24.0.129
--webserver_interface=127.24.0.129
--webserver_port=0
--tserver_master_addrs=127.24.0.190:34415
--builtin_ntp_servers=127.24.0.148:34891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 01:22:59.605075 23375 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:22:59.605347 23375 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:22:59.605373 23375 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:22:59.606499 23375 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:22:59.606537 23375 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:22:59.606561 23375 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.0.129
I20260812 01:22:59.607882 23375 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:34891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786497534195564-24578-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786497534195564-24578-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.0.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786497534195564-24578-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.0.129
--webserver_port=0
--tserver_master_addrs=127.24.0.190:34415
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23375
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.0.129
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786497534195564-24578-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:22:59.608143 23375 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:22:59.608458 23375 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:22:59.614228 23382 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:22:59.614228 23384 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:22:59.614262 23381 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:22:59.614466 23375 server_base.cc:1034] running on GCE node
I20260812 01:22:59.614740 23375 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:22:59.615018 23375 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:22:59.616142 23375 hybrid_clock.cc:648] HybridClock initialized: now 1786497779616118 us; error 38 us; skew 500 ppm
I20260812 01:22:59.617478 23375 webserver.cc:460] Webserver started at http://127.24.0.129:33537/ using document root <none> and password file <none>
I20260812 01:22:59.617745 23375 fs_manager.cc:359] Metadata directory not provided
I20260812 01:22:59.617792 23375 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:22:59.617941 23375 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:22:59.618986 23375 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786497534195564-24578-0/minicluster-data/ts-0/data/instance:
uuid: "8477b2abf53548c5b5bdfcebc3565cc9"
format_stamp: "Formatted at 2026-08-12 01:22:59 on dist-test-slave-t3q3"
server_key: "52f3ace1215cbd88c8079056e5aa636d"
server_key_iv: "d1cdf460b9d75c13c0f44b001e96ef50"
server_key_version: "encryptionkey@0"
I20260812 01:22:59.619365 23375 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786497534195564-24578-0/minicluster-data/ts-0/wal/instance:
uuid: "8477b2abf53548c5b5bdfcebc3565cc9"
format_stamp: "Formatted at 2026-08-12 01:22:59 on dist-test-slave-t3q3"
server_key: "52f3ace1215cbd88c8079056e5aa636d"
server_key_iv: "d1cdf460b9d75c13c0f44b001e96ef50"
server_key_version: "encryptionkey@0"
I20260812 01:22:59.620801 23375 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 01:22:59.621593 23392 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:22:59.621764 23375 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 01:22:59.621870 23375 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786497534195564-24578-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786497534195564-24578-0/minicluster-data/ts-0/wal
uuid: "8477b2abf53548c5b5bdfcebc3565cc9"
format_stamp: "Formatted at 2026-08-12 01:22:59 on dist-test-slave-t3q3"
server_key: "52f3ace1215cbd88c8079056e5aa636d"
server_key_iv: "d1cdf460b9d75c13c0f44b001e96ef50"
server_key_version: "encryptionkey@0"
I20260812 01:22:59.621948 23375 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786497534195564-24578-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786497534195564-24578-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786497534195564-24578-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:22:59.642356 23375 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:22:59.642700 23375 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:22:59.642867 23375 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:22:59.643126 23375 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:22:59.643478 23375 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:22:59.643514 23375 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:22:59.643538 23375 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:22:59.643551 23375 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:22:59.649904 23375 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.129:44273
I20260812 01:22:59.650027 23566 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.129:44273 every 8 connection(s)
I20260812 01:22:59.650328 23375 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786497534195564-24578-0/minicluster-data/ts-0/data/info.pb
I20260812 01:22:59.655740 23342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:59.651138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.129:55399 (local address 127.24.0.190:34415)
0812 01:22:59.651231 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0812 01:22:59.651234 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:22:59.651434 (+ 200us) server_negotiation.cc:407] Connection header received
0812 01:22:59.651991 (+ 557us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:22:59.651992 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:22:59.652005 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:22:59.652120 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:22:59.652824 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:59.653356 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:59.654351 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:59.654567 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:59.654646 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:22:59.655438 (+ 792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:22:59.655439 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:22:59.655440 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:22:59.655445 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:22:59.655460 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:22:59.655465 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:22:59.655538 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 01:22:59.655633 (+ 95us) server_negotiation.cc:299] Negotiation successful
0812 01:22:59.655660 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 01:22:59.655959 23569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:59.651038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:34415 (local address 127.24.0.129:55399)
0812 01:22:59.651310 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0812 01:22:59.651344 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0812 01:22:59.651879 (+ 535us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:22:59.652193 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:22:59.652197 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:22:59.652214 (+ 17us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:22:59.652720 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:22:59.652724 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:22:59.653464 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:22:59.653465 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:22:59.654248 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:22:59.654251 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:22:59.654655 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:22:59.654656 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:22:59.654791 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:22:59.655294 (+ 503us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 01:22:59.655295 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:22:59.655323 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 01:22:59.655347 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:22:59.655541 (+ 194us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:22:59.655546 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:22:59.655549 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 01:22:59.655634 (+ 85us) client_negotiation.cc:240] Negotiation successful
0812 01:22:59.655734 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":93,"threads_started":1}
I20260812 01:22:59.656137 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 23375
I20260812 01:22:59.656250 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786497534195564-24578-0/minicluster-data/ts-0/wal/instance
I20260812 01:22:59.656484 24578 external_mini_cluster.cc:1428] Setting key 78d986cb0b7697a2e22dba7ccf804947
I20260812 01:22:59.656502 23567 heartbeater.cc:347] Connected to a master server at 127.24.0.190:34415
I20260812 01:22:59.656615 23567 heartbeater.cc:464] Registering TS with master...
I20260812 01:22:59.656829 23567 heartbeater.cc:511] Master 127.24.0.190:34415 requested a full tablet report, sending...
I20260812 01:22:59.657336 23282 ts_manager.cc:194] Registered new tserver with Master: 8477b2abf53548c5b5bdfcebc3565cc9 (127.24.0.129:44273)
I20260812 01:22:59.657595 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786497534195564-24578-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786497534195564-24578-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786497534195564-24578-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786497534195564-24578-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.0.130:0
--local_ip_for_outbound_sockets=127.24.0.130
--webserver_interface=127.24.0.130
--webserver_port=0
--tserver_master_addrs=127.24.0.190:34415
--builtin_ntp_servers=127.24.0.148:34891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 01:22:59.658075 23282 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.0.129:55399
W20260812 01:22:59.743065 23570 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:22:59.743266 23570 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:22:59.743287 23570 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:22:59.744499 23570 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:22:59.744541 23570 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:22:59.744567 23570 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.0.130
I20260812 01:22:59.745896 23570 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:34891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786497534195564-24578-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786497534195564-24578-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.0.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786497534195564-24578-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.0.130
--webserver_port=0
--tserver_master_addrs=127.24.0.190:34415
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23570
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.0.130
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786497534195564-24578-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:22:59.746166 23570 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:22:59.746477 23570 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:22:59.751677 23579 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:22:59.751741 23578 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:22:59.751838 23582 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:22:59.751825 23570 server_base.cc:1034] running on GCE node
I20260812 01:22:59.752182 23570 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:22:59.752453 23570 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:22:59.753595 23570 hybrid_clock.cc:648] HybridClock initialized: now 1786497779753582 us; error 29 us; skew 500 ppm
I20260812 01:22:59.754974 23570 webserver.cc:460] Webserver started at http://127.24.0.130:42557/ using document root <none> and password file <none>
I20260812 01:22:59.755245 23570 fs_manager.cc:359] Metadata directory not provided
I20260812 01:22:59.755295 23570 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:22:59.755472 23570 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:22:59.756646 23570 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786497534195564-24578-0/minicluster-data/ts-1/data/instance:
uuid: "0f230d166f8b4bcf8d9cedf07de7732b"
format_stamp: "Formatted at 2026-08-12 01:22:59 on dist-test-slave-t3q3"
server_key: "4e1ead9d43d23f33e1ffd1982a1538e9"
server_key_iv: "1ddb1b4dbcd6b517990418797d652ad4"
server_key_version: "encryptionkey@0"
I20260812 01:22:59.757006 23570 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786497534195564-24578-0/minicluster-data/ts-1/wal/instance:
uuid: "0f230d166f8b4bcf8d9cedf07de7732b"
format_stamp: "Formatted at 2026-08-12 01:22:59 on dist-test-slave-t3q3"
server_key: "4e1ead9d43d23f33e1ffd1982a1538e9"
server_key_iv: "1ddb1b4dbcd6b517990418797d652ad4"
server_key_version: "encryptionkey@0"
I20260812 01:22:59.758347 23570 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 01:22:59.759075 23593 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:22:59.759258 23570 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 01:22:59.759359 23570 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786497534195564-24578-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786497534195564-24578-0/minicluster-data/ts-1/wal
uuid: "0f230d166f8b4bcf8d9cedf07de7732b"
format_stamp: "Formatted at 2026-08-12 01:22:59 on dist-test-slave-t3q3"
server_key: "4e1ead9d43d23f33e1ffd1982a1538e9"
server_key_iv: "1ddb1b4dbcd6b517990418797d652ad4"
server_key_version: "encryptionkey@0"
I20260812 01:22:59.759439 23570 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786497534195564-24578-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786497534195564-24578-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786497534195564-24578-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:22:59.774029 23570 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:22:59.774330 23570 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:22:59.774485 23570 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:22:59.774719 23570 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:22:59.775051 23570 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:22:59.775115 23570 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:22:59.775153 23570 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:22:59.775179 23570 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:22:59.781252 23570 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.130:39141
I20260812 01:22:59.781296 23764 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.130:39141 every 8 connection(s)
I20260812 01:22:59.781651 23570 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786497534195564-24578-0/minicluster-data/ts-1/data/info.pb
I20260812 01:22:59.783555 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 23570
I20260812 01:22:59.783676 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786497534195564-24578-0/minicluster-data/ts-1/wal/instance
I20260812 01:22:59.783883 24578 external_mini_cluster.cc:1428] Setting key 643487b769f81519cbd5fbb2003f12c3
I20260812 01:22:59.784874 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786497534195564-24578-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786497534195564-24578-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786497534195564-24578-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786497534195564-24578-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.0.131:0
--local_ip_for_outbound_sockets=127.24.0.131
--webserver_interface=127.24.0.131
--webserver_port=0
--tserver_master_addrs=127.24.0.190:34415
--builtin_ntp_servers=127.24.0.148:34891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 01:22:59.787590 23342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:59.782486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.130:51945 (local address 127.24.0.190:34415)
0812 01:22:59.782613 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0812 01:22:59.782616 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:22:59.782854 (+ 238us) server_negotiation.cc:407] Connection header received
0812 01:22:59.783424 (+ 570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:22:59.783425 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:22:59.783440 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:22:59.783505 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:22:59.784558 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:59.785111 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:59.786116 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:59.786330 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:59.786410 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:22:59.787306 (+ 896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:22:59.787308 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:22:59.787308 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:22:59.787313 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:22:59.787328 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:22:59.787332 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:22:59.787404 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 01:22:59.787486 (+ 82us) server_negotiation.cc:299] Negotiation successful
0812 01:22:59.787512 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 01:22:59.787863 23769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:59.782455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:34415 (local address 127.24.0.130:51945)
0812 01:22:59.782711 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0812 01:22:59.782746 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0812 01:22:59.783290 (+ 544us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:22:59.783600 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:22:59.783606 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:22:59.783616 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:22:59.784381 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:22:59.784387 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:22:59.785221 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:22:59.785222 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:22:59.786014 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:22:59.786017 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:22:59.786433 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:22:59.786434 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:22:59.786586 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:22:59.787154 (+ 568us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 01:22:59.787154 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:22:59.787183 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 01:22:59.787206 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:22:59.787407 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:22:59.787414 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:22:59.787417 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 01:22:59.787489 (+ 72us) client_negotiation.cc:240] Negotiation successful
0812 01:22:59.787613 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":102,"threads_started":1}
I20260812 01:22:59.788379 23766 heartbeater.cc:347] Connected to a master server at 127.24.0.190:34415
I20260812 01:22:59.788483 23766 heartbeater.cc:464] Registering TS with master...
I20260812 01:22:59.788686 23766 heartbeater.cc:511] Master 127.24.0.190:34415 requested a full tablet report, sending...
I20260812 01:22:59.789052 23282 ts_manager.cc:194] Registered new tserver with Master: 0f230d166f8b4bcf8d9cedf07de7732b (127.24.0.130:39141)
I20260812 01:22:59.789423 23282 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.0.130:51945
W20260812 01:22:59.868758 23770 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:22:59.868960 23770 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:22:59.868983 23770 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:22:59.870146 23770 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:22:59.870195 23770 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:22:59.870220 23770 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.0.131
I20260812 01:22:59.871563 23770 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:34891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786497534195564-24578-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786497534195564-24578-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.0.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786497534195564-24578-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.0.131
--webserver_port=0
--tserver_master_addrs=127.24.0.190:34415
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23770
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.0.131
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786497534195564-24578-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:22:59.871809 23770 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:22:59.872143 23770 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:22:59.877223 23781 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:22:59.877235 23779 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:22:59.877256 23778 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:22:59.877413 23770 server_base.cc:1034] running on GCE node
I20260812 01:22:59.877678 23770 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:22:59.877895 23770 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:22:59.879034 23770 hybrid_clock.cc:648] HybridClock initialized: now 1786497779879011 us; error 38 us; skew 500 ppm
I20260812 01:22:59.880447 23770 webserver.cc:460] Webserver started at http://127.24.0.131:37861/ using document root <none> and password file <none>
I20260812 01:22:59.880750 23770 fs_manager.cc:359] Metadata directory not provided
I20260812 01:22:59.880802 23770 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:22:59.880980 23770 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:22:59.882053 23770 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786497534195564-24578-0/minicluster-data/ts-2/data/instance:
uuid: "a01078ed1f4b498da3e34dd32fade905"
format_stamp: "Formatted at 2026-08-12 01:22:59 on dist-test-slave-t3q3"
server_key: "9bdd2d3045eccc164ae13deda1a7b4a9"
server_key_iv: "7350037d9ce1bb000480630254fdd932"
server_key_version: "encryptionkey@0"
I20260812 01:22:59.882424 23770 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786497534195564-24578-0/minicluster-data/ts-2/wal/instance:
uuid: "a01078ed1f4b498da3e34dd32fade905"
format_stamp: "Formatted at 2026-08-12 01:22:59 on dist-test-slave-t3q3"
server_key: "9bdd2d3045eccc164ae13deda1a7b4a9"
server_key_iv: "7350037d9ce1bb000480630254fdd932"
server_key_version: "encryptionkey@0"
I20260812 01:22:59.883749 23770 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 01:22:59.884634 23788 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:22:59.884868 23770 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 01:22:59.884943 23770 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786497534195564-24578-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786497534195564-24578-0/minicluster-data/ts-2/wal
uuid: "a01078ed1f4b498da3e34dd32fade905"
format_stamp: "Formatted at 2026-08-12 01:22:59 on dist-test-slave-t3q3"
server_key: "9bdd2d3045eccc164ae13deda1a7b4a9"
server_key_iv: "7350037d9ce1bb000480630254fdd932"
server_key_version: "encryptionkey@0"
I20260812 01:22:59.885030 23770 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786497534195564-24578-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786497534195564-24578-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786497534195564-24578-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:22:59.913130 23770 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:22:59.913487 23770 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:22:59.913663 23770 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:22:59.913928 23770 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:22:59.914290 23770 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:22:59.914350 23770 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:22:59.914387 23770 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:22:59.914414 23770 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:22:59.920379 23770 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.131:46499
I20260812 01:22:59.920425 23960 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.131:46499 every 8 connection(s)
I20260812 01:22:59.920770 23770 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786497534195564-24578-0/minicluster-data/ts-2/data/info.pb
I20260812 01:22:59.921625 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 23770
I20260812 01:22:59.921742 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786497534195564-24578-0/minicluster-data/ts-2/wal/instance
I20260812 01:22:59.921983 24578 external_mini_cluster.cc:1428] Setting key b1f7071a6fc6e63c60cb17c78b8d9e83
I20260812 01:22:59.926467 23342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:59.921562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.131:41887 (local address 127.24.0.190:34415)
0812 01:22:59.921641 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0812 01:22:59.921644 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:22:59.921982 (+ 338us) server_negotiation.cc:407] Connection header received
0812 01:22:59.922613 (+ 631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:22:59.922614 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:22:59.922626 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:22:59.922673 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:22:59.923365 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:59.923869 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:59.924969 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:59.925178 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:59.925286 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:22:59.926175 (+ 889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:22:59.926177 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:22:59.926177 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:22:59.926182 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:22:59.926197 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:22:59.926201 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:22:59.926254 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 01:22:59.926380 (+ 126us) server_negotiation.cc:299] Negotiation successful
0812 01:22:59.926405 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 01:22:59.926707 23967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:59.921552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:34415 (local address 127.24.0.131:41887)
0812 01:22:59.921859 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0812 01:22:59.921891 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0812 01:22:59.922503 (+ 612us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:22:59.922743 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:22:59.922747 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:22:59.922757 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:22:59.923272 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:22:59.923276 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:22:59.923999 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:22:59.924000 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:22:59.924860 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:22:59.924862 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:22:59.925303 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:22:59.925304 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:22:59.925452 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:22:59.925981 (+ 529us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 01:22:59.925981 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:22:59.926012 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 01:22:59.926060 (+ 48us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:22:59.926282 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:22:59.926288 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:22:59.926291 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 01:22:59.926360 (+ 69us) client_negotiation.cc:240] Negotiation successful
0812 01:22:59.926459 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":113,"threads_started":1}
I20260812 01:22:59.927183 23965 heartbeater.cc:347] Connected to a master server at 127.24.0.190:34415
I20260812 01:22:59.927302 23965 heartbeater.cc:464] Registering TS with master...
I20260812 01:22:59.927495 23965 heartbeater.cc:511] Master 127.24.0.190:34415 requested a full tablet report, sending...
I20260812 01:22:59.927985 23282 ts_manager.cc:194] Registered new tserver with Master: a01078ed1f4b498da3e34dd32fade905 (127.24.0.131:46499)
I20260812 01:22:59.928404 23282 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.0.131:41887
I20260812 01:22:59.933838 24578 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 01:22:59.938541 23342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:59.935304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53334 (local address 127.24.0.190:34415)
0812 01:22:59.935383 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0812 01:22:59.935386 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:22:59.935486 (+ 100us) server_negotiation.cc:407] Connection header received
0812 01:22:59.935567 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:22:59.935568 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:22:59.935579 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:22:59.935662 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:22:59.935841 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:59.936412 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:59.937220 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:59.937409 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:59.937483 (+ 74us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:22:59.938233 (+ 750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:22:59.938235 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:22:59.938235 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:22:59.938240 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:22:59.938255 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:22:59.938266 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:22:59.938367 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0812 01:22:59.938435 (+ 68us) server_negotiation.cc:299] Negotiation successful
0812 01:22:59.938447 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 01:22:59.940378 23282 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53334:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 01:22:59.941978 23282 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 01:22:59.948454 23981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:59.944336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40956 (local address 127.24.0.130:39141)
0812 01:22:59.944538 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0812 01:22:59.944543 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 01:22:59.944663 (+ 120us) server_negotiation.cc:407] Connection header received
0812 01:22:59.944737 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:22:59.944740 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:22:59.944754 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:22:59.944824 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:22:59.945112 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:59.945809 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:59.947226 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:59.947773 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:59.947870 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:22:59.947970 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0812 01:22:59.948309 (+ 339us) server_negotiation.cc:299] Negotiation successful
0812 01:22:59.948354 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":88,"threads_started":1}
I20260812 01:22:59.948493 23977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:59.944142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.131:46499 (local address 127.0.0.1:36236)
0812 01:22:59.944263 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0812 01:22:59.944303 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 01:22:59.944369 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:22:59.944807 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:22:59.944811 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:22:59.944813 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:22:59.944892 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:22:59.944896 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:22:59.946217 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:22:59.946218 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:22:59.947263 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:22:59.947265 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:22:59.948172 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:22:59.948173 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:22:59.948239 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:22:59.948243 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 01:22:59.948277 (+ 34us) client_negotiation.cc:240] Negotiation successful
0812 01:22:59.948365 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":75,"threads_started":1}
I20260812 01:22:59.948494 23978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:59.944193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.129:44273 (local address 127.0.0.1:45056)
0812 01:22:59.944587 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0812 01:22:59.944598 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 01:22:59.944666 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:22:59.945020 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:22:59.945021 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:22:59.945023 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:22:59.945068 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:22:59.945070 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:22:59.946321 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:22:59.946322 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:22:59.947231 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:22:59.947234 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:22:59.948261 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:22:59.948262 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:22:59.948313 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:22:59.948317 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 01:22:59.948365 (+ 48us) client_negotiation.cc:240] Negotiation successful
0812 01:22:59.948398 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":86,"threads_started":1}
I20260812 01:22:59.948683 23984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:59.944418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45056 (local address 127.24.0.129:44273)
0812 01:22:59.944587 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0812 01:22:59.944592 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 01:22:59.944730 (+ 138us) server_negotiation.cc:407] Connection header received
0812 01:22:59.944778 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:22:59.944781 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:22:59.944866 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:22:59.944947 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:22:59.945185 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:59.946245 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:59.947305 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:59.948045 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:59.948141 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:22:59.948229 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0812 01:22:59.948545 (+ 316us) server_negotiation.cc:299] Negotiation successful
0812 01:22:59.948593 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":81,"threads_started":1}
I20260812 01:22:59.948922 23983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:59.944395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36236 (local address 127.24.0.131:46499)
0812 01:22:59.944630 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0812 01:22:59.944634 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 01:22:59.944644 (+ 10us) server_negotiation.cc:407] Connection header received
0812 01:22:59.944686 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:22:59.944688 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:22:59.944706 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:22:59.944801 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:22:59.944986 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:59.946125 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:59.947353 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:22:59.948045 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:22:59.948653 (+ 608us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:22:59.948752 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 01:22:59.948779 (+ 27us) server_negotiation.cc:299] Negotiation successful
0812 01:22:59.948824 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":206,"thread_start_us":99,"threads_started":1}
I20260812 01:22:59.948495 23980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:22:59.944148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.130:39141 (local address 127.0.0.1:40956)
0812 01:22:59.944586 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0812 01:22:59.944598 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 01:22:59.944666 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:22:59.944883 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:22:59.944884 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:22:59.944887 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:22:59.944947 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:22:59.944948 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:22:59.945901 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:22:59.945903 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:22:59.947134 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:22:59.947137 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:22:59.948039 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:22:59.948040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:22:59.948181 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:22:59.948192 (+ 11us) client_negotiation.cc:769] Sending connection context
0812 01:22:59.948287 (+ 95us) client_negotiation.cc:240] Negotiation successful
0812 01:22:59.948357 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":418,"mutex_wait_us":26,"thread_start_us":68,"threads_started":1}
I20260812 01:22:59.949540 23860 tablet_service.cc:1467] Processing CreateTablet for tablet 550ebeb328d84f4c84585ece815b3dc2 (DEFAULT_TABLE table=test-table [id=ed30d9cb8ede48079424e6f8e8ee22c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:22:59.949539 23471 tablet_service.cc:1467] Processing CreateTablet for tablet 550ebeb328d84f4c84585ece815b3dc2 (DEFAULT_TABLE table=test-table [id=ed30d9cb8ede48079424e6f8e8ee22c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:22:59.949659 23662 tablet_service.cc:1467] Processing CreateTablet for tablet 550ebeb328d84f4c84585ece815b3dc2 (DEFAULT_TABLE table=test-table [id=ed30d9cb8ede48079424e6f8e8ee22c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:22:59.949918 23860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 550ebeb328d84f4c84585ece815b3dc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:22:59.949918 23471 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 550ebeb328d84f4c84585ece815b3dc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:22:59.950066 23662 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 550ebeb328d84f4c84585ece815b3dc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:22:59.952755 23985 tablet_bootstrap.cc:492] T 550ebeb328d84f4c84585ece815b3dc2 P 0f230d166f8b4bcf8d9cedf07de7732b: Bootstrap starting.
I20260812 01:22:59.953301 23987 tablet_bootstrap.cc:492] T 550ebeb328d84f4c84585ece815b3dc2 P 8477b2abf53548c5b5bdfcebc3565cc9: Bootstrap starting.
I20260812 01:22:59.953692 23985 tablet_bootstrap.cc:654] T 550ebeb328d84f4c84585ece815b3dc2 P 0f230d166f8b4bcf8d9cedf07de7732b: Neither blocks nor log segments found. Creating new log.
I20260812 01:22:59.953966 23987 tablet_bootstrap.cc:654] T 550ebeb328d84f4c84585ece815b3dc2 P 8477b2abf53548c5b5bdfcebc3565cc9: Neither blocks nor log segments found. Creating new log.
I20260812 01:22:59.954022 23985 log.cc:824] T 550ebeb328d84f4c84585ece815b3dc2 P 0f230d166f8b4bcf8d9cedf07de7732b: Log is configured to *not* fsync() on all Append() calls
I20260812 01:22:59.954310 23987 log.cc:824] T 550ebeb328d84f4c84585ece815b3dc2 P 8477b2abf53548c5b5bdfcebc3565cc9: Log is configured to *not* fsync() on all Append() calls
I20260812 01:22:59.954661 23991 tablet_bootstrap.cc:492] T 550ebeb328d84f4c84585ece815b3dc2 P a01078ed1f4b498da3e34dd32fade905: Bootstrap starting.
I20260812 01:22:59.954857 23985 tablet_bootstrap.cc:492] T 550ebeb328d84f4c84585ece815b3dc2 P 0f230d166f8b4bcf8d9cedf07de7732b: No bootstrap required, opened a new log
I20260812 01:22:59.954944 23985 ts_tablet_manager.cc:1397] T 550ebeb328d84f4c84585ece815b3dc2 P 0f230d166f8b4bcf8d9cedf07de7732b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 01:22:59.955085 23987 tablet_bootstrap.cc:492] T 550ebeb328d84f4c84585ece815b3dc2 P 8477b2abf53548c5b5bdfcebc3565cc9: No bootstrap required, opened a new log
I20260812 01:22:59.955165 23987 ts_tablet_manager.cc:1397] T 550ebeb328d84f4c84585ece815b3dc2 P 8477b2abf53548c5b5bdfcebc3565cc9: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 01:22:59.955318 23991 tablet_bootstrap.cc:654] T 550ebeb328d84f4c84585ece815b3dc2 P a01078ed1f4b498da3e34dd32fade905: Neither blocks nor log segments found. Creating new log.
I20260812 01:22:59.955669 23991 log.cc:824] T 550ebeb328d84f4c84585ece815b3dc2 P a01078ed1f4b498da3e34dd32fade905: Log is configured to *not* fsync() on all Append() calls
I20260812 01:22:59.956544 23991 tablet_bootstrap.cc:492] T 550ebeb328d84f4c84585ece815b3dc2 P a01078ed1f4b498da3e34dd32fade905: No bootstrap required, opened a new log
I20260812 01:22:59.956619 23991 ts_tablet_manager.cc:1397] T 550ebeb328d84f4c84585ece815b3dc2 P a01078ed1f4b498da3e34dd32fade905: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 01:22:59.957352 23987 raft_consensus.cc:348] T 550ebeb328d84f4c84585ece815b3dc2 P 8477b2abf53548c5b5bdfcebc3565cc9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f230d166f8b4bcf8d9cedf07de7732b" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39141 } } peers { permanent_uuid: "a01078ed1f4b498da3e34dd32fade905" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 46499 } } peers { permanent_uuid: "8477b2abf53548c5b5bdfcebc3565cc9" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 44273 } }
I20260812 01:22:59.957511 23987 raft_consensus.cc:374] T 550ebeb328d84f4c84585ece815b3dc2 P 8477b2abf53548c5b5bdfcebc3565cc9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:22:59.957557 23987 raft_consensus.cc:729] T 550ebeb328d84f4c84585ece815b3dc2 P 8477b2abf53548c5b5bdfcebc3565cc9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8477b2abf53548c5b5bdfcebc3565cc9, State: Initialized, Role: FOLLOWER
I20260812 01:22:59.957671 23987 consensus_queue.cc:260] T 550ebeb328d84f4c84585ece815b3dc2 P 8477b2abf53548c5b5bdfcebc3565cc9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f230d166f8b4bcf8d9cedf07de7732b" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39141 } } peers { permanent_uuid: "a01078ed1f4b498da3e34dd32fade905" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 46499 } } peers { permanent_uuid: "8477b2abf53548c5b5bdfcebc3565cc9" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 44273 } }
I20260812 01:22:59.957965 23567 heartbeater.cc:503] Master 127.24.0.190:34415 was elected leader, sending a full tablet report...
I20260812 01:22:59.957962 23987 ts_tablet_manager.cc:1428] T 550ebeb328d84f4c84585ece815b3dc2 P 8477b2abf53548c5b5bdfcebc3565cc9: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 01:22:59.958101 23985 raft_consensus.cc:348] T 550ebeb328d84f4c84585ece815b3dc2 P 0f230d166f8b4bcf8d9cedf07de7732b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f230d166f8b4bcf8d9cedf07de7732b" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39141 } } peers { permanent_uuid: "a01078ed1f4b498da3e34dd32fade905" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 46499 } } peers { permanent_uuid: "8477b2abf53548c5b5bdfcebc3565cc9" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 44273 } }
I20260812 01:22:59.958250 23985 raft_consensus.cc:374] T 550ebeb328d84f4c84585ece815b3dc2 P 0f230d166f8b4bcf8d9cedf07de7732b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:22:59.958292 23985 raft_consensus.cc:729] T 550ebeb328d84f4c84585ece815b3dc2 P 0f230d166f8b4bcf8d9cedf07de7732b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f230d166f8b4bcf8d9cedf07de7732b, State: Initialized, Role: FOLLOWER
I20260812 01:22:59.958792 23985 consensus_queue.cc:260] T 550ebeb328d84f4c84585ece815b3dc2 P 0f230d166f8b4bcf8d9cedf07de7732b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f230d166f8b4bcf8d9cedf07de7732b" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39141 } } peers { permanent_uuid: "a01078ed1f4b498da3e34dd32fade905" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 46499 } } peers { permanent_uuid: "8477b2abf53548c5b5bdfcebc3565cc9" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 44273 } }
I20260812 01:22:59.959184 23766 heartbeater.cc:503] Master 127.24.0.190:34415 was elected leader, sending a full tablet report...
I20260812 01:22:59.959286 23985 ts_tablet_manager.cc:1428] T 550ebeb328d84f4c84585ece815b3dc2 P 0f230d166f8b4bcf8d9cedf07de7732b: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260812 01:22:59.959205 23991 raft_consensus.cc:348] T 550ebeb328d84f4c84585ece815b3dc2 P a01078ed1f4b498da3e34dd32fade905 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f230d166f8b4bcf8d9cedf07de7732b" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39141 } } peers { permanent_uuid: "a01078ed1f4b498da3e34dd32fade905" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 46499 } } peers { permanent_uuid: "8477b2abf53548c5b5bdfcebc3565cc9" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 44273 } }
I20260812 01:22:59.959363 23991 raft_consensus.cc:374] T 550ebeb328d84f4c84585ece815b3dc2 P a01078ed1f4b498da3e34dd32fade905 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:22:59.959399 23991 raft_consensus.cc:729] T 550ebeb328d84f4c84585ece815b3dc2 P a01078ed1f4b498da3e34dd32fade905 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a01078ed1f4b498da3e34dd32fade905, State: Initialized, Role: FOLLOWER
I20260812 01:22:59.959514 23991 consensus_queue.cc:260] T 550ebeb328d84f4c84585ece815b3dc2 P a01078ed1f4b498da3e34dd32fade905 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f230d166f8b4bcf8d9cedf07de7732b" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39141 } } peers { permanent_uuid: "a01078ed1f4b498da3e34dd32fade905" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 46499 } } peers { permanent_uuid: "8477b2abf53548c5b5bdfcebc3565cc9" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 44273 } }
I20260812 01:22:59.959822 23991 ts_tablet_manager.cc:1428] T 550ebeb328d84f4c84585ece815b3dc2 P a01078ed1f4b498da3e34dd32fade905: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 01:22:59.959944 23965 heartbeater.cc:503] Master 127.24.0.190:34415 was elected leader, sending a full tablet report...
W20260812 01:23:00.032251 23768 tablet.cc:2369] T 550ebeb328d84f4c84585ece815b3dc2 P 0f230d166f8b4bcf8d9cedf07de7732b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 01:23:00.122685 24003 raft_consensus.cc:482] T 550ebeb328d84f4c84585ece815b3dc2 P 0f230d166f8b4bcf8d9cedf07de7732b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 01:23:00.122821 24003 raft_consensus.cc:504] T 550ebeb328d84f4c84585ece815b3dc2 P 0f230d166f8b4bcf8d9cedf07de7732b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f230d166f8b4bcf8d9cedf07de7732b" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39141 } } peers { permanent_uuid: "a01078ed1f4b498da3e34dd32fade905" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 46499 } } peers { permanent_uuid: "8477b2abf53548c5b5bdfcebc3565cc9" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 44273 } }
I20260812 01:23:00.123241 24003 leader_election.cc:288] T 550ebeb328d84f4c84585ece815b3dc2 P 0f230d166f8b4bcf8d9cedf07de7732b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a01078ed1f4b498da3e34dd32fade905 (127.24.0.131:46499), 8477b2abf53548c5b5bdfcebc3565cc9 (127.24.0.129:44273)
I20260812 01:23:00.126803 23984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:00.123523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.130:53925 (local address 127.24.0.129:44273)
0812 01:23:00.123582 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:00.123584 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:00.123777 (+ 193us) server_negotiation.cc:407] Connection header received
0812 01:23:00.123814 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:00.123815 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:00.123825 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:00.123871 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:23:00.124032 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:00.124772 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:00.126068 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:00.126418 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:00.126519 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:00.126527 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:00.126694 (+ 167us) server_negotiation.cc:299] Negotiation successful
0812 01:23:00.126738 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 01:23:00.126811 24008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:00.123474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.129:44273 (local address 127.24.0.130:53925)
0812 01:23:00.123670 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0812 01:23:00.123678 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 01:23:00.123751 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:23:00.123896 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:23:00.123897 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:23:00.123899 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:23:00.123964 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:23:00.123965 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:23:00.124893 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:23:00.124894 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:23:00.125957 (+ 1063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:23:00.125960 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:23:00.126542 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:23:00.126543 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:23:00.126622 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:00.126626 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 01:23:00.126686 (+ 60us) client_negotiation.cc:240] Negotiation successful
0812 01:23:00.126724 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"mutex_wait_us":13,"thread_start_us":116,"threads_started":1}
I20260812 01:23:00.126904 23983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:00.123544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.130:44479 (local address 127.24.0.131:46499)
0812 01:23:00.123596 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:00.123599 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:00.123612 (+ 13us) server_negotiation.cc:407] Connection header received
0812 01:23:00.123645 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:00.123646 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:00.123657 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:00.123720 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:23:00.123938 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:00.124769 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:00.126026 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:00.126412 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:00.126470 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:00.126504 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:00.126785 (+ 281us) server_negotiation.cc:299] Negotiation successful
0812 01:23:00.126827 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260812 01:23:00.126955 23769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:00.123412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.131:46499 (local address 127.24.0.130:44479)
0812 01:23:00.123500 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0812 01:23:00.123513 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 01:23:00.123568 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:23:00.123750 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:23:00.123751 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:23:00.123753 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:23:00.123836 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:23:00.123838 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:23:00.124862 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:23:00.124864 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:23:00.125937 (+ 1073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:23:00.125940 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:23:00.126497 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:23:00.126499 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:23:00.126693 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:00.126696 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 01:23:00.126779 (+ 83us) client_negotiation.cc:240] Negotiation successful
0812 01:23:00.126821 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260812 01:23:00.127224 23500 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "550ebeb328d84f4c84585ece815b3dc2" candidate_uuid: "0f230d166f8b4bcf8d9cedf07de7732b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8477b2abf53548c5b5bdfcebc3565cc9" is_pre_election: true
I20260812 01:23:00.127368 23500 raft_consensus.cc:2454] T 550ebeb328d84f4c84585ece815b3dc2 P 8477b2abf53548c5b5bdfcebc3565cc9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f230d166f8b4bcf8d9cedf07de7732b in term 0.
I20260812 01:23:00.127619 23595 leader_election.cc:302] T 550ebeb328d84f4c84585ece815b3dc2 P 0f230d166f8b4bcf8d9cedf07de7732b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f230d166f8b4bcf8d9cedf07de7732b, 8477b2abf53548c5b5bdfcebc3565cc9; no voters:
I20260812 01:23:00.127600 23890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "550ebeb328d84f4c84585ece815b3dc2" candidate_uuid: "0f230d166f8b4bcf8d9cedf07de7732b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a01078ed1f4b498da3e34dd32fade905" is_pre_election: true
I20260812 01:23:00.127767 23890 raft_consensus.cc:2454] T 550ebeb328d84f4c84585ece815b3dc2 P a01078ed1f4b498da3e34dd32fade905 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f230d166f8b4bcf8d9cedf07de7732b in term 0.
I20260812 01:23:00.127774 24003 raft_consensus.cc:2781] T 550ebeb328d84f4c84585ece815b3dc2 P 0f230d166f8b4bcf8d9cedf07de7732b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 01:23:00.127831 24003 raft_consensus.cc:482] T 550ebeb328d84f4c84585ece815b3dc2 P 0f230d166f8b4bcf8d9cedf07de7732b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 01:23:00.127871 24003 raft_consensus.cc:3037] T 550ebeb328d84f4c84585ece815b3dc2 P 0f230d166f8b4bcf8d9cedf07de7732b [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:23:00.128695 24003 raft_consensus.cc:504] T 550ebeb328d84f4c84585ece815b3dc2 P 0f230d166f8b4bcf8d9cedf07de7732b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f230d166f8b4bcf8d9cedf07de7732b" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39141 } } peers { permanent_uuid: "a01078ed1f4b498da3e34dd32fade905" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 46499 } } peers { permanent_uuid: "8477b2abf53548c5b5bdfcebc3565cc9" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 44273 } }
I20260812 01:23:00.128856 24003 leader_election.cc:288] T 550ebeb328d84f4c84585ece815b3dc2 P 0f230d166f8b4bcf8d9cedf07de7732b [CANDIDATE]: Term 1 election: Requested vote from peers a01078ed1f4b498da3e34dd32fade905 (127.24.0.131:46499), 8477b2abf53548c5b5bdfcebc3565cc9 (127.24.0.129:44273)
I20260812 01:23:00.128988 23890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "550ebeb328d84f4c84585ece815b3dc2" candidate_uuid: "0f230d166f8b4bcf8d9cedf07de7732b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a01078ed1f4b498da3e34dd32fade905"
I20260812 01:23:00.129047 23500 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "550ebeb328d84f4c84585ece815b3dc2" candidate_uuid: "0f230d166f8b4bcf8d9cedf07de7732b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8477b2abf53548c5b5bdfcebc3565cc9"
I20260812 01:23:00.129089 23890 raft_consensus.cc:3037] T 550ebeb328d84f4c84585ece815b3dc2 P a01078ed1f4b498da3e34dd32fade905 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:23:00.129163 23500 raft_consensus.cc:3037] T 550ebeb328d84f4c84585ece815b3dc2 P 8477b2abf53548c5b5bdfcebc3565cc9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:23:00.129874 23890 raft_consensus.cc:2454] T 550ebeb328d84f4c84585ece815b3dc2 P a01078ed1f4b498da3e34dd32fade905 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f230d166f8b4bcf8d9cedf07de7732b in term 1.
I20260812 01:23:00.129943 23500 raft_consensus.cc:2454] T 550ebeb328d84f4c84585ece815b3dc2 P 8477b2abf53548c5b5bdfcebc3565cc9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f230d166f8b4bcf8d9cedf07de7732b in term 1.
I20260812 01:23:00.130136 23594 leader_election.cc:302] T 550ebeb328d84f4c84585ece815b3dc2 P 0f230d166f8b4bcf8d9cedf07de7732b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f230d166f8b4bcf8d9cedf07de7732b, a01078ed1f4b498da3e34dd32fade905; no voters:
I20260812 01:23:00.130224 24003 raft_consensus.cc:2781] T 550ebeb328d84f4c84585ece815b3dc2 P 0f230d166f8b4bcf8d9cedf07de7732b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:23:00.130394 24003 raft_consensus.cc:686] T 550ebeb328d84f4c84585ece815b3dc2 P 0f230d166f8b4bcf8d9cedf07de7732b [term 1 LEADER]: Becoming Leader. State: Replica: 0f230d166f8b4bcf8d9cedf07de7732b, State: Running, Role: LEADER
I20260812 01:23:00.130505 24003 consensus_queue.cc:237] T 550ebeb328d84f4c84585ece815b3dc2 P 0f230d166f8b4bcf8d9cedf07de7732b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f230d166f8b4bcf8d9cedf07de7732b" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39141 } } peers { permanent_uuid: "a01078ed1f4b498da3e34dd32fade905" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 46499 } } peers { permanent_uuid: "8477b2abf53548c5b5bdfcebc3565cc9" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 44273 } }
I20260812 01:23:00.131347 23282 catalog_manager.cc:5495] T 550ebeb328d84f4c84585ece815b3dc2 P 0f230d166f8b4bcf8d9cedf07de7732b reported cstate change: term changed from 0 to 1, leader changed from <none> to 0f230d166f8b4bcf8d9cedf07de7732b (127.24.0.130). New cstate: current_term: 1 leader_uuid: "0f230d166f8b4bcf8d9cedf07de7732b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f230d166f8b4bcf8d9cedf07de7732b" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 39141 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a01078ed1f4b498da3e34dd32fade905" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 46499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8477b2abf53548c5b5bdfcebc3565cc9" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 44273 } health_report { overall_health: UNKNOWN } } }
W20260812 01:23:00.151103 23568 tablet.cc:2369] T 550ebeb328d84f4c84585ece815b3dc2 P 8477b2abf53548c5b5bdfcebc3565cc9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 01:23:00.171336 23966 tablet.cc:2369] T 550ebeb328d84f4c84585ece815b3dc2 P a01078ed1f4b498da3e34dd32fade905: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 01:23:00.174846 23981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:00.172350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40970 (local address 127.24.0.130:39141)
0812 01:23:00.172412 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:00.172414 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:00.172424 (+ 10us) server_negotiation.cc:407] Connection header received
0812 01:23:00.172509 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:00.172510 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:00.172542 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:00.172627 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:23:00.172760 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:00.173280 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:00.174037 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:00.174236 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:00.174315 (+ 79us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:00.174462 (+ 147us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:23:00.174562 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:23:00.174670 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:00.174731 (+ 61us) server_negotiation.cc:299] Negotiation successful
0812 01:23:00.174739 (+ 8us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260812 01:23:00.176931 23500 raft_consensus.cc:1261] T 550ebeb328d84f4c84585ece815b3dc2 P 8477b2abf53548c5b5bdfcebc3565cc9 [term 1 FOLLOWER]: Refusing update from remote peer 0f230d166f8b4bcf8d9cedf07de7732b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:23:00.176931 23890 raft_consensus.cc:1261] T 550ebeb328d84f4c84585ece815b3dc2 P a01078ed1f4b498da3e34dd32fade905 [term 1 FOLLOWER]: Refusing update from remote peer 0f230d166f8b4bcf8d9cedf07de7732b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:23:00.177338 24010 consensus_queue.cc:1035] T 550ebeb328d84f4c84585ece815b3dc2 P 0f230d166f8b4bcf8d9cedf07de7732b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a01078ed1f4b498da3e34dd32fade905" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 46499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:23:00.177492 24003 consensus_queue.cc:1035] T 550ebeb328d84f4c84585ece815b3dc2 P 0f230d166f8b4bcf8d9cedf07de7732b [LEADER]: Connected to new peer: Peer: permanent_uuid: "8477b2abf53548c5b5bdfcebc3565cc9" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 44273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:23:00.179752 24016 mvcc.cc:205] Tried to move back new op lower bound from 7317494907601252352 to 7317494907415187456. Current Snapshot: MvccSnapshot[applied={T|T < 7317494907601252352}]
I20260812 01:23:00.180586 24018 mvcc.cc:205] Tried to move back new op lower bound from 7317494907601252352 to 7317494907415187456. Current Snapshot: MvccSnapshot[applied={T|T < 7317494907601252352}]
I20260812 01:23:00.182824 23281 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:53334:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:23:00.182934 23281 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:53334:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:23:00.183809 23281 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fc10eb8737fd4f7188eb24e60b7d2083: Sending DeleteTablet for 3 replicas of tablet 550ebeb328d84f4c84585ece815b3dc2
I20260812 01:23:00.184191 23471 tablet_service.cc:1514] Processing DeleteTablet for tablet 550ebeb328d84f4c84585ece815b3dc2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:23:00 UTC) from {username='slave'} at 127.0.0.1:45056
I20260812 01:23:00.184197 23662 tablet_service.cc:1514] Processing DeleteTablet for tablet 550ebeb328d84f4c84585ece815b3dc2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:23:00 UTC) from {username='slave'} at 127.0.0.1:40956
I20260812 01:23:00.184253 23860 tablet_service.cc:1514] Processing DeleteTablet for tablet 550ebeb328d84f4c84585ece815b3dc2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:23:00 UTC) from {username='slave'} at 127.0.0.1:36236
I20260812 01:23:00.184518 24034 tablet_replica.cc:330] T 550ebeb328d84f4c84585ece815b3dc2 P a01078ed1f4b498da3e34dd32fade905: stopping tablet replica
I20260812 01:23:00.184608 24031 tablet_replica.cc:330] T 550ebeb328d84f4c84585ece815b3dc2 P 0f230d166f8b4bcf8d9cedf07de7732b: stopping tablet replica
I20260812 01:23:00.184679 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 23375
I20260812 01:23:00.184705 24031 raft_consensus.cc:2229] T 550ebeb328d84f4c84585ece815b3dc2 P 0f230d166f8b4bcf8d9cedf07de7732b [term 1 LEADER]: Raft consensus shutting down.
I20260812 01:23:00.184722 24034 raft_consensus.cc:2229] T 550ebeb328d84f4c84585ece815b3dc2 P a01078ed1f4b498da3e34dd32fade905 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 01:23:00.187569 24034 raft_consensus.cc:2258] T 550ebeb328d84f4c84585ece815b3dc2 P a01078ed1f4b498da3e34dd32fade905 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:23:00.199637 24031 raft_consensus.cc:2258] T 550ebeb328d84f4c84585ece815b3dc2 P 0f230d166f8b4bcf8d9cedf07de7732b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:23:00.200166 24034 ts_tablet_manager.cc:1905] T 550ebeb328d84f4c84585ece815b3dc2 P a01078ed1f4b498da3e34dd32fade905: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:23:00.200315 24031 ts_tablet_manager.cc:1905] T 550ebeb328d84f4c84585ece815b3dc2 P 0f230d166f8b4bcf8d9cedf07de7732b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:23:00.202366 24034 ts_tablet_manager.cc:1918] T 550ebeb328d84f4c84585ece815b3dc2 P a01078ed1f4b498da3e34dd32fade905: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 01:23:00.202517 24031 ts_tablet_manager.cc:1918] T 550ebeb328d84f4c84585ece815b3dc2 P 0f230d166f8b4bcf8d9cedf07de7732b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 01:23:00.202548 24034 log.cc:1196] T 550ebeb328d84f4c84585ece815b3dc2 P a01078ed1f4b498da3e34dd32fade905: Deleting WAL directory at /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786497534195564-24578-0/minicluster-data/ts-2/wal/wals/550ebeb328d84f4c84585ece815b3dc2
I20260812 01:23:00.202649 24031 log.cc:1196] T 550ebeb328d84f4c84585ece815b3dc2 P 0f230d166f8b4bcf8d9cedf07de7732b: Deleting WAL directory at /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786497534195564-24578-0/minicluster-data/ts-1/wal/wals/550ebeb328d84f4c84585ece815b3dc2
I20260812 01:23:00.202919 24034 ts_tablet_manager.cc:1939] T 550ebeb328d84f4c84585ece815b3dc2 P a01078ed1f4b498da3e34dd32fade905: Deleting consensus metadata
I20260812 01:23:00.202965 24031 ts_tablet_manager.cc:1939] T 550ebeb328d84f4c84585ece815b3dc2 P 0f230d166f8b4bcf8d9cedf07de7732b: Deleting consensus metadata
I20260812 01:23:00.203538 23262 catalog_manager.cc:4841] TS 0f230d166f8b4bcf8d9cedf07de7732b (127.24.0.130:39141): tablet 550ebeb328d84f4c84585ece815b3dc2 (table test-table [id=ed30d9cb8ede48079424e6f8e8ee22c0]) successfully deleted
I20260812 01:23:00.203619 23258 catalog_manager.cc:4841] TS a01078ed1f4b498da3e34dd32fade905 (127.24.0.131:46499): tablet 550ebeb328d84f4c84585ece815b3dc2 (table test-table [id=ed30d9cb8ede48079424e6f8e8ee22c0]) successfully deleted
W20260812 01:23:00.204288 23261 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.0.129:44273 (error 108)
I20260812 01:23:00.204541 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 23570
I20260812 01:23:00.209022 23980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:00.204741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.129:44273 (local address 127.0.0.1:45060)
0812 01:23:00.208929 (+ 4188us) negotiation.cc:107] Waiting for socket to connect
0812 01:23:00.208985 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.129:44273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4162}
W20260812 01:23:00.209189 23261 catalog_manager.cc:4584] TS 8477b2abf53548c5b5bdfcebc3565cc9 (127.24.0.129:44273): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 550ebeb328d84f4c84585ece815b3dc2: Network error: Client connection negotiation failed: client connection to 127.24.0.129:44273: connect: Connection refused (error 111)
I20260812 01:23:00.222127 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 23770
I20260812 01:23:00.236159 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 23236
2026-08-12T01:23:00Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (930 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786497534195564-24578-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 01:23:00 dist-test-slave-t3q3 krb5kdc[24040](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 01:23:00 dist-test-slave-t3q3 krb5kdc[24040](info): set up 2 sockets
Aug 12 01:23:00 dist-test-slave-t3q3 krb5kdc[24040](info): commencing operation
krb5kdc: starting...
W20260812 01:23:01.499495 24578 mini_kdc.cc:121] Time spent starting KDC: real 1.239s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786497534195564-24578-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786497534195564-24578-0/krb5kdc/test-user.keytab.
Aug 12 01:23:01 dist-test-slave-t3q3 krb5kdc[24040](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497781, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T01:23:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:23:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.0.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.0.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.0.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab.
Entry for principal kudu/127.24.0.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab.
I20260812 01:23:01.624727 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786497534195564-24578-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786497534195564-24578-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786497534195564-24578-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.190:34917
--webserver_interface=127.24.0.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab
--principal=kudu/127.24.0.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:40569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.0.190:34917
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:23:01.709908 24060 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:23:01.710227 24060 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:23:01.710250 24060 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:23:01.711428 24060 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:23:01.711476 24060 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:23:01.711490 24060 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:23:01.711506 24060 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:23:01.711520 24060 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:23:01.713164 24060 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:40569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786497534195564-24578-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786497534195564-24578-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.0.190:34917
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab
--principal=kudu/127.24.0.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.190:34917
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.0.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24060
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786497534195564-24578-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:23:01.713438 24060 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:23:01.713680 24060 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:23:01.718887 24071 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:23:01.718840 24068 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:23:01.718900 24069 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:23:01.718937 24060 server_base.cc:1034] running on GCE node
I20260812 01:23:01.719417 24060 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:23:01.719686 24060 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:23:01.720829 24060 hybrid_clock.cc:648] HybridClock initialized: now 1786497781720815 us; error 30 us; skew 500 ppm
Aug 12 01:23:01 dist-test-slave-t3q3 krb5kdc[24040](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497781, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:23:01.723498 24060 init.cc:377] Logged in from keytab as kudu/127.24.0.190@KRBTEST.COM (short username kudu)
I20260812 01:23:01.724107 24060 webserver.cc:460] Webserver started at http://127.24.0.190:44913/ using document root <none> and password file <none>
I20260812 01:23:01.724360 24060 fs_manager.cc:359] Metadata directory not provided
I20260812 01:23:01.724406 24060 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:23:01.724498 24060 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:23:01.725569 24060 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786497534195564-24578-0/minicluster-data/master-0/data/instance:
uuid: "9a9cf8116f71488c99dad819f55bf5b8"
format_stamp: "Formatted at 2026-08-12 01:23:01 on dist-test-slave-t3q3"
server_key: "26a8c4b739380a30e6735de8d2b3d489"
server_key_iv: "b06b8325e20c9de431bebdf114d33d06"
server_key_version: "encryptionkey@0"
I20260812 01:23:01.725889 24060 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786497534195564-24578-0/minicluster-data/master-0/wal/instance:
uuid: "9a9cf8116f71488c99dad819f55bf5b8"
format_stamp: "Formatted at 2026-08-12 01:23:01 on dist-test-slave-t3q3"
server_key: "26a8c4b739380a30e6735de8d2b3d489"
server_key_iv: "b06b8325e20c9de431bebdf114d33d06"
server_key_version: "encryptionkey@0"
I20260812 01:23:01.727279 24060 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 01:23:01.728088 24094 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:23:01.728297 24060 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 01:23:01.728399 24060 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786497534195564-24578-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786497534195564-24578-0/minicluster-data/master-0/wal
uuid: "9a9cf8116f71488c99dad819f55bf5b8"
format_stamp: "Formatted at 2026-08-12 01:23:01 on dist-test-slave-t3q3"
server_key: "26a8c4b739380a30e6735de8d2b3d489"
server_key_iv: "b06b8325e20c9de431bebdf114d33d06"
server_key_version: "encryptionkey@0"
I20260812 01:23:01.728495 24060 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786497534195564-24578-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786497534195564-24578-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786497534195564-24578-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:23:01.736270 24060 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:23:01.738698 24060 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:23:01.738868 24060 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:23:01.743371 24060 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.190:34917
I20260812 01:23:01.743409 24171 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.190:34917 every 8 connection(s)
I20260812 01:23:01.743808 24060 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
I20260812 01:23:01.744730 24172 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:23:01.748231 24172 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a9cf8116f71488c99dad819f55bf5b8: Bootstrap starting.
I20260812 01:23:01.749029 24172 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9a9cf8116f71488c99dad819f55bf5b8: Neither blocks nor log segments found. Creating new log.
I20260812 01:23:01.749356 24172 log.cc:824] T 00000000000000000000000000000000 P 9a9cf8116f71488c99dad819f55bf5b8: Log is configured to *not* fsync() on all Append() calls
I20260812 01:23:01.750159 24172 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a9cf8116f71488c99dad819f55bf5b8: No bootstrap required, opened a new log
I20260812 01:23:01.750672 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 24060
I20260812 01:23:01.750767 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786497534195564-24578-0/minicluster-data/master-0/wal/instance
I20260812 01:23:01.750984 24578 external_mini_cluster.cc:1428] Setting key 0c82ee9d1312201acc5977c2f899fea3
I20260812 01:23:01.752460 24172 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9a9cf8116f71488c99dad819f55bf5b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a9cf8116f71488c99dad819f55bf5b8" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 34917 } }
I20260812 01:23:01.752640 24172 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9a9cf8116f71488c99dad819f55bf5b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:23:01.752681 24172 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9a9cf8116f71488c99dad819f55bf5b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a9cf8116f71488c99dad819f55bf5b8, State: Initialized, Role: FOLLOWER
I20260812 01:23:01.752880 24172 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9a9cf8116f71488c99dad819f55bf5b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a9cf8116f71488c99dad819f55bf5b8" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 34917 } }
I20260812 01:23:01.753008 24172 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9a9cf8116f71488c99dad819f55bf5b8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:23:01.753055 24172 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9a9cf8116f71488c99dad819f55bf5b8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:23:01.753113 24172 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9a9cf8116f71488c99dad819f55bf5b8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:23:01.754142 24172 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9a9cf8116f71488c99dad819f55bf5b8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a9cf8116f71488c99dad819f55bf5b8" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 34917 } }
I20260812 01:23:01.754276 24172 leader_election.cc:302] T 00000000000000000000000000000000 P 9a9cf8116f71488c99dad819f55bf5b8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9a9cf8116f71488c99dad819f55bf5b8; no voters:
I20260812 01:23:01.754456 24172 leader_election.cc:288] T 00000000000000000000000000000000 P 9a9cf8116f71488c99dad819f55bf5b8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 01:23:01.754508 24178 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9a9cf8116f71488c99dad819f55bf5b8 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:23:01.754738 24172 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9a9cf8116f71488c99dad819f55bf5b8 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:23:01.754770 24178 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9a9cf8116f71488c99dad819f55bf5b8 [term 1 LEADER]: Becoming Leader. State: Replica: 9a9cf8116f71488c99dad819f55bf5b8, State: Running, Role: LEADER
I20260812 01:23:01.754884 24178 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9a9cf8116f71488c99dad819f55bf5b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a9cf8116f71488c99dad819f55bf5b8" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 34917 } }
I20260812 01:23:01.755306 24178 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a9cf8116f71488c99dad819f55bf5b8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9a9cf8116f71488c99dad819f55bf5b8. Latest consensus state: current_term: 1 leader_uuid: "9a9cf8116f71488c99dad819f55bf5b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a9cf8116f71488c99dad819f55bf5b8" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 34917 } } }
I20260812 01:23:01.755316 24180 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a9cf8116f71488c99dad819f55bf5b8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9a9cf8116f71488c99dad819f55bf5b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a9cf8116f71488c99dad819f55bf5b8" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 34917 } } }
I20260812 01:23:01.755430 24178 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a9cf8116f71488c99dad819f55bf5b8 [sys.catalog]: This master's current role is: LEADER
I20260812 01:23:01.755439 24180 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a9cf8116f71488c99dad819f55bf5b8 [sys.catalog]: This master's current role is: LEADER
I20260812 01:23:01.755715 24192 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:23:01.756273 24192 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:23:01.758159 24192 catalog_manager.cc:1347] Generated new cluster ID: 7525b3834b0f474c951c23850845718e
I20260812 01:23:01.758221 24192 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 01:23:01 dist-test-slave-t3q3 krb5kdc[24040](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497781, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
I20260812 01:23:01.763669 24177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:01.751679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53100 (local address 127.24.0.190:34917)
0812 01:23:01.751851 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:01.751859 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:01.751884 (+ 25us) server_negotiation.cc:407] Connection header received
0812 01:23:01.752438 (+ 554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:01.752445 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:01.752496 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:01.752618 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:23:01.753484 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:01.754532 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:01.755617 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:01.756264 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:01.756381 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:01.759607 (+ 3226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:23:01.759611 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:23:01.759612 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:23:01.759625 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:23:01.761617 (+ 1992us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:23:01.762050 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:23:01.762053 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:23:01.762054 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:23:01.762187 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:23:01.762420 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:23:01.762421 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:23:01.762421 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:23:01.762730 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:23:01.763251 (+ 521us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:01.763318 (+ 67us) server_negotiation.cc:299] Negotiation successful
0812 01:23:01.763459 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":77,"threads_started":1}
I20260812 01:23:01.766516 24192 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:23:01.766997 24192 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:23:01.771327 24192 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9a9cf8116f71488c99dad819f55bf5b8: Generated new TSK 0
I20260812 01:23:01.771523 24192 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.0.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.0.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.0.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.129.keytab.
Entry for principal kudu/127.24.0.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.129.keytab.
I20260812 01:23:01.801105 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786497534195564-24578-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786497534195564-24578-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786497534195564-24578-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786497534195564-24578-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.0.129:0
--local_ip_for_outbound_sockets=127.24.0.129
--webserver_interface=127.24.0.129
--webserver_port=0
--tserver_master_addrs=127.24.0.190:34917
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.129.keytab
--principal=kudu/127.24.0.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:40569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:23:01.885545 24211 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:23:01.885792 24211 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:23:01.885833 24211 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:23:01.887043 24211 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:23:01.887110 24211 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:23:01.887157 24211 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.0.129
I20260812 01:23:01.888652 24211 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:40569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786497534195564-24578-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786497534195564-24578-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.129.keytab
--principal=kudu/127.24.0.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.0.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786497534195564-24578-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.0.129
--webserver_port=0
--tserver_master_addrs=127.24.0.190:34917
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24211
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.0.129
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786497534195564-24578-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:23:01.888931 24211 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:23:01.889185 24211 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:23:01.894639 24222 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:23:01.894647 24218 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:23:01.894646 24217 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:23:01.894835 24211 server_base.cc:1034] running on GCE node
I20260812 01:23:01.895159 24211 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:23:01.895413 24211 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:23:01.896576 24211 hybrid_clock.cc:648] HybridClock initialized: now 1786497781896548 us; error 46 us; skew 500 ppm
Aug 12 01:23:01 dist-test-slave-t3q3 krb5kdc[24040](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497781, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:23:01.899374 24211 init.cc:377] Logged in from keytab as kudu/127.24.0.129@KRBTEST.COM (short username kudu)
I20260812 01:23:01.899844 24211 webserver.cc:460] Webserver started at http://127.24.0.129:40731/ using document root <none> and password file <none>
I20260812 01:23:01.900154 24211 fs_manager.cc:359] Metadata directory not provided
I20260812 01:23:01.900229 24211 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:23:01.900395 24211 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:23:01.901546 24211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786497534195564-24578-0/minicluster-data/ts-0/data/instance:
uuid: "d0120f0908884dcdad627e94725c4823"
format_stamp: "Formatted at 2026-08-12 01:23:01 on dist-test-slave-t3q3"
server_key: "b5bc37006a913918651dae8c9f84cc40"
server_key_iv: "37d867356130fddbc6871ca0c84c3507"
server_key_version: "encryptionkey@0"
I20260812 01:23:01.901939 24211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786497534195564-24578-0/minicluster-data/ts-0/wal/instance:
uuid: "d0120f0908884dcdad627e94725c4823"
format_stamp: "Formatted at 2026-08-12 01:23:01 on dist-test-slave-t3q3"
server_key: "b5bc37006a913918651dae8c9f84cc40"
server_key_iv: "37d867356130fddbc6871ca0c84c3507"
server_key_version: "encryptionkey@0"
I20260812 01:23:01.903364 24211 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 01:23:01.904258 24235 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:23:01.904433 24211 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:23:01.904536 24211 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786497534195564-24578-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786497534195564-24578-0/minicluster-data/ts-0/wal
uuid: "d0120f0908884dcdad627e94725c4823"
format_stamp: "Formatted at 2026-08-12 01:23:01 on dist-test-slave-t3q3"
server_key: "b5bc37006a913918651dae8c9f84cc40"
server_key_iv: "37d867356130fddbc6871ca0c84c3507"
server_key_version: "encryptionkey@0"
I20260812 01:23:01.904623 24211 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786497534195564-24578-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786497534195564-24578-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786497534195564-24578-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:23:01.922472 24211 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:23:01.924666 24211 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:23:01.924836 24211 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:23:01.925091 24211 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:23:01.925436 24211 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:23:01.925493 24211 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:23:01.925530 24211 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:23:01.925581 24211 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:23:01.932008 24211 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.129:34935
I20260812 01:23:01.932075 24408 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.129:34935 every 8 connection(s)
I20260812 01:23:01.932476 24211 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786497534195564-24578-0/minicluster-data/ts-0/data/info.pb
I20260812 01:23:01.937072 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 24211
I20260812 01:23:01.937209 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786497534195564-24578-0/minicluster-data/ts-0/wal/instance
I20260812 01:23:01.937404 24578 external_mini_cluster.cc:1428] Setting key 9f961d2a40bb13324f3784a6b5aee66a
Aug 12 01:23:01 dist-test-slave-t3q3 krb5kdc[24040](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497781, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.129@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
I20260812 01:23:01.942775 24177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:01.933317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.129:51847 (local address 127.24.0.190:34917)
0812 01:23:01.933432 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:01.933434 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:01.933622 (+ 188us) server_negotiation.cc:407] Connection header received
0812 01:23:01.934200 (+ 578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:01.934201 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:01.934218 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:01.934272 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:23:01.935288 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:01.935852 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:01.937024 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:01.937315 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:01.937402 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:01.939784 (+ 2382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:23:01.939786 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:23:01.939787 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:23:01.939802 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:23:01.941328 (+ 1526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:23:01.941775 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:23:01.941777 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:23:01.941777 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:23:01.941838 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:23:01.942105 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:23:01.942106 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:23:01.942107 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:23:01.942224 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:23:01.942298 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:01.942586 (+ 288us) server_negotiation.cc:299] Negotiation successful
0812 01:23:01.942680 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260812 01:23:01.942989 24413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:01.933268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:34917 (local address 127.24.0.129:51847)
0812 01:23:01.933470 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0812 01:23:01.933523 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 01:23:01.934083 (+ 560us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:23:01.934361 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:23:01.934364 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:23:01.934956 (+ 592us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:23:01.935166 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:23:01.935179 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:23:01.936030 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:23:01.936031 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:23:01.936915 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:23:01.936917 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:23:01.937412 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:23:01.937413 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:23:01.937600 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:01.938240 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:23:01.938250 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:23:01.939663 (+ 1413us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:23:01.941433 (+ 1770us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:23:01.941441 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:23:01.941443 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:23:01.941677 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:23:01.941905 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:23:01.941906 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:23:01.941906 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:23:01.942025 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:23:01.942301 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:23:01.942309 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:23:01.942451 (+ 142us) client_negotiation.cc:769] Sending connection context
0812 01:23:01.942562 (+ 111us) client_negotiation.cc:240] Negotiation successful
0812 01:23:01.942735 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":93,"threads_started":1}
I20260812 01:23:01.943638 24409 heartbeater.cc:347] Connected to a master server at 127.24.0.190:34917
I20260812 01:23:01.943769 24409 heartbeater.cc:464] Registering TS with master...
I20260812 01:23:01.943995 24409 heartbeater.cc:511] Master 127.24.0.190:34917 requested a full tablet report, sending...
I20260812 01:23:01.944639 24122 ts_manager.cc:194] Registered new tserver with Master: d0120f0908884dcdad627e94725c4823 (127.24.0.129:34935)
I20260812 01:23:01.945601 24122 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.0.129@KRBTEST.COM'} at 127.24.0.129:51847
WARNING: no policy specified for kudu/127.24.0.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.0.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.0.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.130.keytab.
Entry for principal kudu/127.24.0.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.130.keytab.
I20260812 01:23:01.965229 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786497534195564-24578-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786497534195564-24578-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786497534195564-24578-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786497534195564-24578-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.0.130:0
--local_ip_for_outbound_sockets=127.24.0.130
--webserver_interface=127.24.0.130
--webserver_port=0
--tserver_master_addrs=127.24.0.190:34917
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.130.keytab
--principal=kudu/127.24.0.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:40569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:23:02.049719 24417 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:23:02.049937 24417 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:23:02.049961 24417 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:23:02.051136 24417 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:23:02.051190 24417 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:23:02.051218 24417 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.0.130
I20260812 01:23:02.052712 24417 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:40569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786497534195564-24578-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786497534195564-24578-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.130.keytab
--principal=kudu/127.24.0.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.0.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786497534195564-24578-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.0.130
--webserver_port=0
--tserver_master_addrs=127.24.0.190:34917
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24417
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.0.130
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786497534195564-24578-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:23:02.052963 24417 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:23:02.053231 24417 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:23:02.058409 24426 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:23:02.058399 24430 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:23:02.058408 24428 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:23:02.058797 24417 server_base.cc:1034] running on GCE node
I20260812 01:23:02.059053 24417 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:23:02.059324 24417 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:23:02.060514 24417 hybrid_clock.cc:648] HybridClock initialized: now 1786497782060490 us; error 51 us; skew 500 ppm
Aug 12 01:23:02 dist-test-slave-t3q3 krb5kdc[24040](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497782, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:23:02.063274 24417 init.cc:377] Logged in from keytab as kudu/127.24.0.130@KRBTEST.COM (short username kudu)
I20260812 01:23:02.063782 24417 webserver.cc:460] Webserver started at http://127.24.0.130:36085/ using document root <none> and password file <none>
I20260812 01:23:02.064091 24417 fs_manager.cc:359] Metadata directory not provided
I20260812 01:23:02.064182 24417 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:23:02.064348 24417 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:23:02.065431 24417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786497534195564-24578-0/minicluster-data/ts-1/data/instance:
uuid: "06b7eba5a9434d7f97de86825b7efdff"
format_stamp: "Formatted at 2026-08-12 01:23:02 on dist-test-slave-t3q3"
server_key: "7bf3bb245f559c9ab4c59639d29a734e"
server_key_iv: "3a2963529d95f1c941874cffcdc0951c"
server_key_version: "encryptionkey@0"
I20260812 01:23:02.065822 24417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786497534195564-24578-0/minicluster-data/ts-1/wal/instance:
uuid: "06b7eba5a9434d7f97de86825b7efdff"
format_stamp: "Formatted at 2026-08-12 01:23:02 on dist-test-slave-t3q3"
server_key: "7bf3bb245f559c9ab4c59639d29a734e"
server_key_iv: "3a2963529d95f1c941874cffcdc0951c"
server_key_version: "encryptionkey@0"
I20260812 01:23:02.067191 24417 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 01:23:02.068024 24440 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:23:02.068251 24417 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:23:02.068342 24417 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786497534195564-24578-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786497534195564-24578-0/minicluster-data/ts-1/wal
uuid: "06b7eba5a9434d7f97de86825b7efdff"
format_stamp: "Formatted at 2026-08-12 01:23:02 on dist-test-slave-t3q3"
server_key: "7bf3bb245f559c9ab4c59639d29a734e"
server_key_iv: "3a2963529d95f1c941874cffcdc0951c"
server_key_version: "encryptionkey@0"
I20260812 01:23:02.068392 24417 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786497534195564-24578-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786497534195564-24578-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786497534195564-24578-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:23:02.085480 24417 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:23:02.087935 24417 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:23:02.088112 24417 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:23:02.088357 24417 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:23:02.088699 24417 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:23:02.088737 24417 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:23:02.088759 24417 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:23:02.088771 24417 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:23:02.094928 24417 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.130:35807
I20260812 01:23:02.094974 24639 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.130:35807 every 8 connection(s)
I20260812 01:23:02.095304 24417 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786497534195564-24578-0/minicluster-data/ts-1/data/info.pb
Aug 12 01:23:02 dist-test-slave-t3q3 krb5kdc[24040](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497782, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.130@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
I20260812 01:23:02.101615 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 24417
I20260812 01:23:02.101778 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786497534195564-24578-0/minicluster-data/ts-1/wal/instance
I20260812 01:23:02.102010 24578 external_mini_cluster.cc:1428] Setting key 51d9910e757fb6b09eefbc13f8b05964
I20260812 01:23:02.105079 24177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:02.096092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.130:53053 (local address 127.24.0.190:34917)
0812 01:23:02.096161 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:02.096164 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:02.096361 (+ 197us) server_negotiation.cc:407] Connection header received
0812 01:23:02.096913 (+ 552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:02.096914 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:02.096927 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:02.097001 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:23:02.098018 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:02.098524 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:02.099606 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:02.099817 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:02.099905 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:02.102134 (+ 2229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:23:02.102136 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:23:02.102136 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:23:02.102144 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:23:02.103586 (+ 1442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:23:02.104015 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:23:02.104016 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:23:02.104016 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:23:02.104142 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:23:02.104408 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:23:02.104409 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:23:02.104409 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:23:02.104527 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:23:02.104597 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:02.104889 (+ 292us) server_negotiation.cc:299] Negotiation successful
0812 01:23:02.104987 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 01:23:02.105257 24647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:02.096024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:34917 (local address 127.24.0.130:53053)
0812 01:23:02.096215 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0812 01:23:02.096266 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 01:23:02.096793 (+ 527us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:23:02.097076 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:23:02.097080 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:23:02.097674 (+ 594us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:23:02.097890 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:23:02.097896 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:23:02.098637 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:23:02.098638 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:23:02.099501 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:23:02.099504 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:23:02.099916 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:23:02.099917 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:23:02.100051 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:02.100578 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:23:02.100587 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:23:02.102030 (+ 1443us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:23:02.103691 (+ 1661us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:23:02.103698 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:23:02.103701 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:23:02.103925 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:23:02.104216 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:23:02.104217 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:23:02.104217 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:23:02.104332 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:23:02.104606 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:23:02.104614 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:23:02.104751 (+ 137us) client_negotiation.cc:769] Sending connection context
0812 01:23:02.104854 (+ 103us) client_negotiation.cc:240] Negotiation successful
0812 01:23:02.105016 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":111,"threads_started":1}
I20260812 01:23:02.105727 24645 heartbeater.cc:347] Connected to a master server at 127.24.0.190:34917
I20260812 01:23:02.105841 24645 heartbeater.cc:464] Registering TS with master...
I20260812 01:23:02.106040 24645 heartbeater.cc:511] Master 127.24.0.190:34917 requested a full tablet report, sending...
I20260812 01:23:02.106490 24122 ts_manager.cc:194] Registered new tserver with Master: 06b7eba5a9434d7f97de86825b7efdff (127.24.0.130:35807)
I20260812 01:23:02.107023 24122 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.0.130@KRBTEST.COM'} at 127.24.0.130:53053
WARNING: no policy specified for kudu/127.24.0.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.0.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.0.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.131.keytab.
Entry for principal kudu/127.24.0.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.131.keytab.
I20260812 01:23:02.130241 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786497534195564-24578-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786497534195564-24578-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786497534195564-24578-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786497534195564-24578-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.0.131:0
--local_ip_for_outbound_sockets=127.24.0.131
--webserver_interface=127.24.0.131
--webserver_port=0
--tserver_master_addrs=127.24.0.190:34917
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.131.keytab
--principal=kudu/127.24.0.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:40569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:23:02.214372 24652 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:23:02.214576 24652 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:23:02.214600 24652 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:23:02.215826 24652 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:23:02.215871 24652 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:23:02.215898 24652 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.0.131
I20260812 01:23:02.217383 24652 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:40569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786497534195564-24578-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786497534195564-24578-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.131.keytab
--principal=kudu/127.24.0.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.0.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786497534195564-24578-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.0.131
--webserver_port=0
--tserver_master_addrs=127.24.0.190:34917
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24652
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.0.131
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786497534195564-24578-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:23:02.217645 24652 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:23:02.217895 24652 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:23:02.222945 24662 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:23:02.222918 24665 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:23:02.222980 24663 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:23:02.223186 24652 server_base.cc:1034] running on GCE node
I20260812 01:23:02.223416 24652 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:23:02.223678 24652 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:23:02.224850 24652 hybrid_clock.cc:648] HybridClock initialized: now 1786497782224832 us; error 31 us; skew 500 ppm
Aug 12 01:23:02 dist-test-slave-t3q3 krb5kdc[24040](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497782, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:23:02.227430 24652 init.cc:377] Logged in from keytab as kudu/127.24.0.131@KRBTEST.COM (short username kudu)
I20260812 01:23:02.227875 24652 webserver.cc:460] Webserver started at http://127.24.0.131:45113/ using document root <none> and password file <none>
I20260812 01:23:02.228168 24652 fs_manager.cc:359] Metadata directory not provided
I20260812 01:23:02.228236 24652 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:23:02.228394 24652 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:23:02.229441 24652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786497534195564-24578-0/minicluster-data/ts-2/data/instance:
uuid: "0e588bd328bd493797faf8b912f8b24b"
format_stamp: "Formatted at 2026-08-12 01:23:02 on dist-test-slave-t3q3"
server_key: "240e92e96adbf41cf7fbcd8f84452d52"
server_key_iv: "2b8dca29444cf14b2c953c33a5594135"
server_key_version: "encryptionkey@0"
I20260812 01:23:02.229813 24652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786497534195564-24578-0/minicluster-data/ts-2/wal/instance:
uuid: "0e588bd328bd493797faf8b912f8b24b"
format_stamp: "Formatted at 2026-08-12 01:23:02 on dist-test-slave-t3q3"
server_key: "240e92e96adbf41cf7fbcd8f84452d52"
server_key_iv: "2b8dca29444cf14b2c953c33a5594135"
server_key_version: "encryptionkey@0"
I20260812 01:23:02.231294 24652 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 01:23:02.232208 24675 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:23:02.232399 24652 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 01:23:02.232501 24652 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786497534195564-24578-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786497534195564-24578-0/minicluster-data/ts-2/wal
uuid: "0e588bd328bd493797faf8b912f8b24b"
format_stamp: "Formatted at 2026-08-12 01:23:02 on dist-test-slave-t3q3"
server_key: "240e92e96adbf41cf7fbcd8f84452d52"
server_key_iv: "2b8dca29444cf14b2c953c33a5594135"
server_key_version: "encryptionkey@0"
I20260812 01:23:02.232597 24652 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786497534195564-24578-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786497534195564-24578-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786497534195564-24578-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:23:02.270164 24652 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:23:02.272733 24652 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:23:02.272922 24652 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:23:02.273200 24652 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:23:02.273545 24652 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:23:02.273600 24652 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:23:02.273648 24652 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:23:02.273679 24652 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:23:02.279564 24652 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.131:38539
I20260812 01:23:02.279649 24865 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.131:38539 every 8 connection(s)
I20260812 01:23:02.280005 24652 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786497534195564-24578-0/minicluster-data/ts-2/data/info.pb
Aug 12 01:23:02 dist-test-slave-t3q3 krb5kdc[24040](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497782, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.131@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
I20260812 01:23:02.286363 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 24652
I20260812 01:23:02.286511 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786497534195564-24578-0/minicluster-data/ts-2/wal/instance
I20260812 01:23:02.286711 24578 external_mini_cluster.cc:1428] Setting key 0e24b8c340f1de36ddd1e7a5ae6f0778
I20260812 01:23:02.289948 24177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:02.280921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.131:40737 (local address 127.24.0.190:34917)
0812 01:23:02.280989 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:02.280991 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:02.281184 (+ 193us) server_negotiation.cc:407] Connection header received
0812 01:23:02.281728 (+ 544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:02.281729 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:02.281741 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:02.281798 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:23:02.282788 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:02.283312 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:02.284406 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:02.284605 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:02.284674 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:02.287020 (+ 2346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:23:02.287022 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:23:02.287022 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:23:02.287030 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:23:02.288627 (+ 1597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:23:02.289075 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:23:02.289076 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:23:02.289076 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:23:02.289142 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:23:02.289411 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:23:02.289411 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:23:02.289412 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:23:02.289516 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:23:02.289567 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:02.289819 (+ 252us) server_negotiation.cc:299] Negotiation successful
0812 01:23:02.289882 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 01:23:02.290252 24870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:02.280894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:34917 (local address 127.24.0.131:40737)
0812 01:23:02.281067 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0812 01:23:02.281107 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 01:23:02.281626 (+ 519us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:23:02.281868 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:23:02.281871 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:23:02.282463 (+ 592us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:23:02.282687 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:23:02.282693 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:23:02.283397 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:23:02.283398 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:23:02.284324 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:23:02.284326 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:23:02.284689 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:23:02.284690 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:23:02.284797 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:02.285303 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:23:02.285312 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:23:02.286920 (+ 1608us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:23:02.288721 (+ 1801us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:23:02.288728 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:23:02.288730 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:23:02.288978 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:23:02.289229 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:23:02.289230 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:23:02.289230 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:23:02.289347 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:23:02.289575 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:23:02.289580 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:23:02.289706 (+ 126us) client_negotiation.cc:769] Sending connection context
0812 01:23:02.289792 (+ 86us) client_negotiation.cc:240] Negotiation successful
0812 01:23:02.289932 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":103,"threads_started":1}
I20260812 01:23:02.290820 24867 heartbeater.cc:347] Connected to a master server at 127.24.0.190:34917
I20260812 01:23:02.290922 24867 heartbeater.cc:464] Registering TS with master...
I20260812 01:23:02.291118 24867 heartbeater.cc:511] Master 127.24.0.190:34917 requested a full tablet report, sending...
I20260812 01:23:02.291532 24122 ts_manager.cc:194] Registered new tserver with Master: 0e588bd328bd493797faf8b912f8b24b (127.24.0.131:38539)
I20260812 01:23:02.291921 24122 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.0.131@KRBTEST.COM'} at 127.24.0.131:40737
I20260812 01:23:02.298653 24578 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 01:23:02.306347 24177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:02.300044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53106 (local address 127.24.0.190:34917)
0812 01:23:02.300156 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:02.300158 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:02.300242 (+ 84us) server_negotiation.cc:407] Connection header received
0812 01:23:02.300284 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:02.300285 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:02.300295 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:02.300383 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:23:02.300925 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:02.301453 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:02.302136 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:02.302341 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:02.302404 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:02.303617 (+ 1213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:23:02.303618 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:23:02.303619 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:23:02.303625 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:23:02.305319 (+ 1694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:23:02.305635 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:23:02.305636 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:23:02.305636 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:23:02.305691 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:23:02.305841 (+ 150us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:23:02.305842 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:23:02.305842 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:23:02.305986 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:23:02.306067 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:02.306169 (+ 102us) server_negotiation.cc:299] Negotiation successful
0812 01:23:02.306232 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260812 01:23:02.308287 24122 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53106:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 01:23:02.309962 24122 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 01:23:02.316799 24895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:02.312808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36850 (local address 127.24.0.129:34935)
0812 01:23:02.313245 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:02.313249 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:02.313259 (+ 10us) server_negotiation.cc:407] Connection header received
0812 01:23:02.313301 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:02.313303 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:02.313318 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:02.313422 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:23:02.313530 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:02.314221 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:02.315632 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:02.316373 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:02.316465 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:02.316532 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:02.316677 (+ 145us) server_negotiation.cc:299] Negotiation successful
0812 01:23:02.316720 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":79,"threads_started":1}
I20260812 01:23:02.316789 24886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:02.312471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.129:34935 (local address 127.0.0.1:36850)
0812 01:23:02.312870 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0812 01:23:02.312882 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 01:23:02.312929 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:23:02.313422 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:23:02.313423 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:23:02.313424 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:23:02.313461 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:23:02.313463 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:23:02.314325 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:23:02.314326 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:23:02.315520 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:23:02.315523 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:23:02.316466 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:23:02.316467 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:23:02.316547 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:02.316561 (+ 14us) client_negotiation.cc:769] Sending connection context
0812 01:23:02.316658 (+ 97us) client_negotiation.cc:240] Negotiation successful
0812 01:23:02.316715 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":74,"threads_started":1}
I20260812 01:23:02.317045 24885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:02.312470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.131:38539 (local address 127.0.0.1:45258)
0812 01:23:02.312655 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0812 01:23:02.312694 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 01:23:02.312768 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:23:02.313258 (+ 490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:23:02.313262 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:23:02.313264 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:23:02.313341 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:23:02.313343 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:23:02.314762 (+ 1419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:23:02.314763 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:23:02.315928 (+ 1165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:23:02.315931 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:23:02.316807 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:23:02.316808 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:23:02.316874 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:02.316879 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 01:23:02.316912 (+ 33us) client_negotiation.cc:240] Negotiation successful
0812 01:23:02.316962 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"mutex_wait_us":35,"thread_start_us":98,"threads_started":1}
I20260812 01:23:02.317250 24894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:02.312772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54202 (local address 127.24.0.130:35807)
0812 01:23:02.312961 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:02.312967 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:02.313084 (+ 117us) server_negotiation.cc:407] Connection header received
0812 01:23:02.313241 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:02.313244 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:02.313261 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:02.313341 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:23:02.313572 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:02.314644 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:02.315913 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:02.316672 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:02.316751 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:02.316815 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:02.317093 (+ 278us) server_negotiation.cc:299] Negotiation successful
0812 01:23:02.317157 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":98,"threads_started":1}
I20260812 01:23:02.317478 24893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:02.312679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45258 (local address 127.24.0.131:38539)
0812 01:23:02.313063 (+ 384us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:02.313067 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:02.313079 (+ 12us) server_negotiation.cc:407] Connection header received
0812 01:23:02.313128 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:02.313131 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:02.313147 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:02.313421 (+ 274us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:23:02.313492 (+ 71us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:02.314621 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:02.316003 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:02.316710 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:02.317236 (+ 526us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:02.317309 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:02.317336 (+ 27us) server_negotiation.cc:299] Negotiation successful
0812 01:23:02.317390 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":101,"threads_started":1}
I20260812 01:23:02.317054 24889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:02.312508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.130:35807 (local address 127.0.0.1:54202)
0812 01:23:02.312999 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0812 01:23:02.313008 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 01:23:02.313092 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:23:02.313425 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:23:02.313426 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:23:02.313429 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:23:02.313505 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:23:02.313507 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:23:02.314765 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:23:02.314766 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:23:02.315824 (+ 1058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:23:02.315827 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:23:02.316844 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:23:02.316844 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:23:02.316908 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:02.316912 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 01:23:02.316959 (+ 47us) client_negotiation.cc:240] Negotiation successful
0812 01:23:02.316992 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":102,"threads_started":1}
I20260812 01:23:02.317956 24308 tablet_service.cc:1467] Processing CreateTablet for tablet 854ecd93e67f49f6b8e8f9e16e4a06be (DEFAULT_TABLE table=test-table [id=e5f84ae72a4e47efb3ed0ef7095aaab4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:23:02.318154 24766 tablet_service.cc:1467] Processing CreateTablet for tablet 854ecd93e67f49f6b8e8f9e16e4a06be (DEFAULT_TABLE table=test-table [id=e5f84ae72a4e47efb3ed0ef7095aaab4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:23:02.318208 24517 tablet_service.cc:1467] Processing CreateTablet for tablet 854ecd93e67f49f6b8e8f9e16e4a06be (DEFAULT_TABLE table=test-table [id=e5f84ae72a4e47efb3ed0ef7095aaab4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:23:02.318367 24308 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 854ecd93e67f49f6b8e8f9e16e4a06be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:23:02.318509 24517 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 854ecd93e67f49f6b8e8f9e16e4a06be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:23:02.318526 24766 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 854ecd93e67f49f6b8e8f9e16e4a06be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:23:02.321328 24896 tablet_bootstrap.cc:492] T 854ecd93e67f49f6b8e8f9e16e4a06be P 0e588bd328bd493797faf8b912f8b24b: Bootstrap starting.
I20260812 01:23:02.321844 24899 tablet_bootstrap.cc:492] T 854ecd93e67f49f6b8e8f9e16e4a06be P d0120f0908884dcdad627e94725c4823: Bootstrap starting.
I20260812 01:23:02.322062 24900 tablet_bootstrap.cc:492] T 854ecd93e67f49f6b8e8f9e16e4a06be P 06b7eba5a9434d7f97de86825b7efdff: Bootstrap starting.
I20260812 01:23:02.322182 24896 tablet_bootstrap.cc:654] T 854ecd93e67f49f6b8e8f9e16e4a06be P 0e588bd328bd493797faf8b912f8b24b: Neither blocks nor log segments found. Creating new log.
I20260812 01:23:02.322793 24900 tablet_bootstrap.cc:654] T 854ecd93e67f49f6b8e8f9e16e4a06be P 06b7eba5a9434d7f97de86825b7efdff: Neither blocks nor log segments found. Creating new log.
I20260812 01:23:02.322795 24896 log.cc:824] T 854ecd93e67f49f6b8e8f9e16e4a06be P 0e588bd328bd493797faf8b912f8b24b: Log is configured to *not* fsync() on all Append() calls
I20260812 01:23:02.323403 24900 log.cc:824] T 854ecd93e67f49f6b8e8f9e16e4a06be P 06b7eba5a9434d7f97de86825b7efdff: Log is configured to *not* fsync() on all Append() calls
I20260812 01:23:02.323624 24896 tablet_bootstrap.cc:492] T 854ecd93e67f49f6b8e8f9e16e4a06be P 0e588bd328bd493797faf8b912f8b24b: No bootstrap required, opened a new log
I20260812 01:23:02.323701 24896 ts_tablet_manager.cc:1397] T 854ecd93e67f49f6b8e8f9e16e4a06be P 0e588bd328bd493797faf8b912f8b24b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 01:23:02.324260 24900 tablet_bootstrap.cc:492] T 854ecd93e67f49f6b8e8f9e16e4a06be P 06b7eba5a9434d7f97de86825b7efdff: No bootstrap required, opened a new log
I20260812 01:23:02.324321 24899 tablet_bootstrap.cc:654] T 854ecd93e67f49f6b8e8f9e16e4a06be P d0120f0908884dcdad627e94725c4823: Neither blocks nor log segments found. Creating new log.
I20260812 01:23:02.324350 24900 ts_tablet_manager.cc:1397] T 854ecd93e67f49f6b8e8f9e16e4a06be P 06b7eba5a9434d7f97de86825b7efdff: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 01:23:02.324939 24899 log.cc:824] T 854ecd93e67f49f6b8e8f9e16e4a06be P d0120f0908884dcdad627e94725c4823: Log is configured to *not* fsync() on all Append() calls
I20260812 01:23:02.325759 24899 tablet_bootstrap.cc:492] T 854ecd93e67f49f6b8e8f9e16e4a06be P d0120f0908884dcdad627e94725c4823: No bootstrap required, opened a new log
I20260812 01:23:02.325839 24899 ts_tablet_manager.cc:1397] T 854ecd93e67f49f6b8e8f9e16e4a06be P d0120f0908884dcdad627e94725c4823: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260812 01:23:02.326071 24896 raft_consensus.cc:348] T 854ecd93e67f49f6b8e8f9e16e4a06be P 0e588bd328bd493797faf8b912f8b24b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0120f0908884dcdad627e94725c4823" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 34935 } } peers { permanent_uuid: "0e588bd328bd493797faf8b912f8b24b" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 38539 } } peers { permanent_uuid: "06b7eba5a9434d7f97de86825b7efdff" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 35807 } }
I20260812 01:23:02.326272 24896 raft_consensus.cc:374] T 854ecd93e67f49f6b8e8f9e16e4a06be P 0e588bd328bd493797faf8b912f8b24b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:23:02.326175 24900 raft_consensus.cc:348] T 854ecd93e67f49f6b8e8f9e16e4a06be P 06b7eba5a9434d7f97de86825b7efdff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0120f0908884dcdad627e94725c4823" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 34935 } } peers { permanent_uuid: "0e588bd328bd493797faf8b912f8b24b" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 38539 } } peers { permanent_uuid: "06b7eba5a9434d7f97de86825b7efdff" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 35807 } }
I20260812 01:23:02.326310 24896 raft_consensus.cc:729] T 854ecd93e67f49f6b8e8f9e16e4a06be P 0e588bd328bd493797faf8b912f8b24b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e588bd328bd493797faf8b912f8b24b, State: Initialized, Role: FOLLOWER
I20260812 01:23:02.326375 24900 raft_consensus.cc:374] T 854ecd93e67f49f6b8e8f9e16e4a06be P 06b7eba5a9434d7f97de86825b7efdff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:23:02.326428 24900 raft_consensus.cc:729] T 854ecd93e67f49f6b8e8f9e16e4a06be P 06b7eba5a9434d7f97de86825b7efdff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06b7eba5a9434d7f97de86825b7efdff, State: Initialized, Role: FOLLOWER
I20260812 01:23:02.326469 24896 consensus_queue.cc:260] T 854ecd93e67f49f6b8e8f9e16e4a06be P 0e588bd328bd493797faf8b912f8b24b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0120f0908884dcdad627e94725c4823" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 34935 } } peers { permanent_uuid: "0e588bd328bd493797faf8b912f8b24b" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 38539 } } peers { permanent_uuid: "06b7eba5a9434d7f97de86825b7efdff" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 35807 } }
I20260812 01:23:02.326555 24900 consensus_queue.cc:260] T 854ecd93e67f49f6b8e8f9e16e4a06be P 06b7eba5a9434d7f97de86825b7efdff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0120f0908884dcdad627e94725c4823" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 34935 } } peers { permanent_uuid: "0e588bd328bd493797faf8b912f8b24b" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 38539 } } peers { permanent_uuid: "06b7eba5a9434d7f97de86825b7efdff" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 35807 } }
I20260812 01:23:02.326788 24896 ts_tablet_manager.cc:1428] T 854ecd93e67f49f6b8e8f9e16e4a06be P 0e588bd328bd493797faf8b912f8b24b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 01:23:02.326853 24645 heartbeater.cc:503] Master 127.24.0.190:34917 was elected leader, sending a full tablet report...
I20260812 01:23:02.326952 24867 heartbeater.cc:503] Master 127.24.0.190:34917 was elected leader, sending a full tablet report...
I20260812 01:23:02.327104 24900 ts_tablet_manager.cc:1428] T 854ecd93e67f49f6b8e8f9e16e4a06be P 06b7eba5a9434d7f97de86825b7efdff: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 01:23:02.328032 24899 raft_consensus.cc:348] T 854ecd93e67f49f6b8e8f9e16e4a06be P d0120f0908884dcdad627e94725c4823 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0120f0908884dcdad627e94725c4823" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 34935 } } peers { permanent_uuid: "0e588bd328bd493797faf8b912f8b24b" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 38539 } } peers { permanent_uuid: "06b7eba5a9434d7f97de86825b7efdff" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 35807 } }
I20260812 01:23:02.328301 24899 raft_consensus.cc:374] T 854ecd93e67f49f6b8e8f9e16e4a06be P d0120f0908884dcdad627e94725c4823 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:23:02.328395 24899 raft_consensus.cc:729] T 854ecd93e67f49f6b8e8f9e16e4a06be P d0120f0908884dcdad627e94725c4823 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0120f0908884dcdad627e94725c4823, State: Initialized, Role: FOLLOWER
I20260812 01:23:02.328550 24899 consensus_queue.cc:260] T 854ecd93e67f49f6b8e8f9e16e4a06be P d0120f0908884dcdad627e94725c4823 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0120f0908884dcdad627e94725c4823" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 34935 } } peers { permanent_uuid: "0e588bd328bd493797faf8b912f8b24b" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 38539 } } peers { permanent_uuid: "06b7eba5a9434d7f97de86825b7efdff" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 35807 } }
I20260812 01:23:02.328831 24899 ts_tablet_manager.cc:1428] T 854ecd93e67f49f6b8e8f9e16e4a06be P d0120f0908884dcdad627e94725c4823: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 01:23:02.328883 24409 heartbeater.cc:503] Master 127.24.0.190:34917 was elected leader, sending a full tablet report...
W20260812 01:23:02.345863 24646 tablet.cc:2369] T 854ecd93e67f49f6b8e8f9e16e4a06be P 06b7eba5a9434d7f97de86825b7efdff: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 01:23:02.373925 24908 raft_consensus.cc:482] T 854ecd93e67f49f6b8e8f9e16e4a06be P d0120f0908884dcdad627e94725c4823 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 01:23:02.374061 24908 raft_consensus.cc:504] T 854ecd93e67f49f6b8e8f9e16e4a06be P d0120f0908884dcdad627e94725c4823 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0120f0908884dcdad627e94725c4823" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 34935 } } peers { permanent_uuid: "0e588bd328bd493797faf8b912f8b24b" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 38539 } } peers { permanent_uuid: "06b7eba5a9434d7f97de86825b7efdff" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 35807 } }
I20260812 01:23:02.374545 24908 leader_election.cc:288] T 854ecd93e67f49f6b8e8f9e16e4a06be P d0120f0908884dcdad627e94725c4823 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e588bd328bd493797faf8b912f8b24b (127.24.0.131:38539), 06b7eba5a9434d7f97de86825b7efdff (127.24.0.130:35807)
I20260812 01:23:02.378803 24893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:02.374980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.129:48629 (local address 127.24.0.131:38539)
0812 01:23:02.375038 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:02.375040 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:02.375050 (+ 10us) server_negotiation.cc:407] Connection header received
0812 01:23:02.375102 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:02.375104 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:02.375115 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:02.375183 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:23:02.375393 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:02.376169 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:02.377791 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:02.378391 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:02.378477 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:02.378496 (+ 19us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:02.378686 (+ 190us) server_negotiation.cc:299] Negotiation successful
0812 01:23:02.378728 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 01:23:02.378803 24894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:02.374856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.129:32925 (local address 127.24.0.130:35807)
0812 01:23:02.374927 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:02.374929 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:02.375110 (+ 181us) server_negotiation.cc:407] Connection header received
0812 01:23:02.375142 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:02.375143 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:02.375157 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:02.375215 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:23:02.375393 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:02.376195 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:02.377791 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:02.378405 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:02.378477 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:02.378487 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:02.378686 (+ 199us) server_negotiation.cc:299] Negotiation successful
0812 01:23:02.378728 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 01:23:02.378816 24413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:02.374750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.131:38539 (local address 127.24.0.129:48629)
0812 01:23:02.374862 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0812 01:23:02.374876 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 01:23:02.374930 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:23:02.375209 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:23:02.375209 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:23:02.375211 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:23:02.375293 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:23:02.375295 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:23:02.376274 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:23:02.376275 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:23:02.377673 (+ 1398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:23:02.377677 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:23:02.378503 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:23:02.378504 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:23:02.378599 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:02.378601 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 01:23:02.378661 (+ 60us) client_negotiation.cc:240] Negotiation successful
0812 01:23:02.378718 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84,"mutex_wait_us":113}
I20260812 01:23:02.378831 24909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:02.374788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.130:35807 (local address 127.24.0.129:32925)
0812 01:23:02.375011 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0812 01:23:02.375023 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 01:23:02.375094 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:23:02.375238 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:23:02.375238 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:23:02.375239 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:23:02.375299 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:23:02.375300 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:23:02.376274 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:23:02.376275 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:23:02.377673 (+ 1398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:23:02.377677 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:23:02.378503 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:23:02.378504 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:23:02.378592 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:02.378596 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 01:23:02.378661 (+ 65us) client_negotiation.cc:240] Negotiation successful
0812 01:23:02.378718 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203,"mutex_wait_us":121,"thread_start_us":147,"threads_started":1}
I20260812 01:23:02.379263 24796 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "854ecd93e67f49f6b8e8f9e16e4a06be" candidate_uuid: "d0120f0908884dcdad627e94725c4823" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e588bd328bd493797faf8b912f8b24b" is_pre_election: true
I20260812 01:23:02.379443 24796 raft_consensus.cc:2454] T 854ecd93e67f49f6b8e8f9e16e4a06be P 0e588bd328bd493797faf8b912f8b24b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d0120f0908884dcdad627e94725c4823 in term 0.
I20260812 01:23:02.379508 24553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "854ecd93e67f49f6b8e8f9e16e4a06be" candidate_uuid: "d0120f0908884dcdad627e94725c4823" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06b7eba5a9434d7f97de86825b7efdff" is_pre_election: true
I20260812 01:23:02.379650 24553 raft_consensus.cc:2454] T 854ecd93e67f49f6b8e8f9e16e4a06be P 06b7eba5a9434d7f97de86825b7efdff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d0120f0908884dcdad627e94725c4823 in term 0.
I20260812 01:23:02.379709 24238 leader_election.cc:302] T 854ecd93e67f49f6b8e8f9e16e4a06be P d0120f0908884dcdad627e94725c4823 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e588bd328bd493797faf8b912f8b24b, d0120f0908884dcdad627e94725c4823; no voters:
I20260812 01:23:02.379849 24908 raft_consensus.cc:2781] T 854ecd93e67f49f6b8e8f9e16e4a06be P d0120f0908884dcdad627e94725c4823 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 01:23:02.379916 24908 raft_consensus.cc:482] T 854ecd93e67f49f6b8e8f9e16e4a06be P d0120f0908884dcdad627e94725c4823 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 01:23:02.379940 24908 raft_consensus.cc:3037] T 854ecd93e67f49f6b8e8f9e16e4a06be P d0120f0908884dcdad627e94725c4823 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:23:02.380676 24908 raft_consensus.cc:504] T 854ecd93e67f49f6b8e8f9e16e4a06be P d0120f0908884dcdad627e94725c4823 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0120f0908884dcdad627e94725c4823" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 34935 } } peers { permanent_uuid: "0e588bd328bd493797faf8b912f8b24b" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 38539 } } peers { permanent_uuid: "06b7eba5a9434d7f97de86825b7efdff" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 35807 } }
I20260812 01:23:02.380818 24908 leader_election.cc:288] T 854ecd93e67f49f6b8e8f9e16e4a06be P d0120f0908884dcdad627e94725c4823 [CANDIDATE]: Term 1 election: Requested vote from peers 0e588bd328bd493797faf8b912f8b24b (127.24.0.131:38539), 06b7eba5a9434d7f97de86825b7efdff (127.24.0.130:35807)
I20260812 01:23:02.381031 24796 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "854ecd93e67f49f6b8e8f9e16e4a06be" candidate_uuid: "d0120f0908884dcdad627e94725c4823" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e588bd328bd493797faf8b912f8b24b"
I20260812 01:23:02.381037 24553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "854ecd93e67f49f6b8e8f9e16e4a06be" candidate_uuid: "d0120f0908884dcdad627e94725c4823" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06b7eba5a9434d7f97de86825b7efdff"
I20260812 01:23:02.381121 24796 raft_consensus.cc:3037] T 854ecd93e67f49f6b8e8f9e16e4a06be P 0e588bd328bd493797faf8b912f8b24b [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:23:02.381119 24553 raft_consensus.cc:3037] T 854ecd93e67f49f6b8e8f9e16e4a06be P 06b7eba5a9434d7f97de86825b7efdff [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:23:02.381814 24796 raft_consensus.cc:2454] T 854ecd93e67f49f6b8e8f9e16e4a06be P 0e588bd328bd493797faf8b912f8b24b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0120f0908884dcdad627e94725c4823 in term 1.
I20260812 01:23:02.381888 24553 raft_consensus.cc:2454] T 854ecd93e67f49f6b8e8f9e16e4a06be P 06b7eba5a9434d7f97de86825b7efdff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0120f0908884dcdad627e94725c4823 in term 1.
I20260812 01:23:02.382133 24236 leader_election.cc:302] T 854ecd93e67f49f6b8e8f9e16e4a06be P d0120f0908884dcdad627e94725c4823 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06b7eba5a9434d7f97de86825b7efdff, d0120f0908884dcdad627e94725c4823; no voters:
I20260812 01:23:02.382256 24908 raft_consensus.cc:2781] T 854ecd93e67f49f6b8e8f9e16e4a06be P d0120f0908884dcdad627e94725c4823 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:23:02.382416 24908 raft_consensus.cc:686] T 854ecd93e67f49f6b8e8f9e16e4a06be P d0120f0908884dcdad627e94725c4823 [term 1 LEADER]: Becoming Leader. State: Replica: d0120f0908884dcdad627e94725c4823, State: Running, Role: LEADER
I20260812 01:23:02.382520 24908 consensus_queue.cc:237] T 854ecd93e67f49f6b8e8f9e16e4a06be P d0120f0908884dcdad627e94725c4823 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0120f0908884dcdad627e94725c4823" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 34935 } } peers { permanent_uuid: "0e588bd328bd493797faf8b912f8b24b" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 38539 } } peers { permanent_uuid: "06b7eba5a9434d7f97de86825b7efdff" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 35807 } }
I20260812 01:23:02.383463 24121 catalog_manager.cc:5495] T 854ecd93e67f49f6b8e8f9e16e4a06be P d0120f0908884dcdad627e94725c4823 reported cstate change: term changed from 0 to 1, leader changed from <none> to d0120f0908884dcdad627e94725c4823 (127.24.0.129). New cstate: current_term: 1 leader_uuid: "d0120f0908884dcdad627e94725c4823" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0120f0908884dcdad627e94725c4823" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 34935 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e588bd328bd493797faf8b912f8b24b" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 38539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06b7eba5a9434d7f97de86825b7efdff" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 35807 } health_report { overall_health: UNKNOWN } } }
I20260812 01:23:02.389662 24895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:02.387398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36852 (local address 127.24.0.129:34935)
0812 01:23:02.387448 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:02.387450 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:02.387460 (+ 10us) server_negotiation.cc:407] Connection header received
0812 01:23:02.387530 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:02.387530 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:02.387547 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:02.387605 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:23:02.387713 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:02.388217 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:02.388930 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:02.389136 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:02.389193 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:02.389367 (+ 174us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:23:02.389454 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:23:02.389511 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:02.389551 (+ 40us) server_negotiation.cc:299] Negotiation successful
0812 01:23:02.389567 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 01:23:02.391335 24553 raft_consensus.cc:1261] T 854ecd93e67f49f6b8e8f9e16e4a06be P 06b7eba5a9434d7f97de86825b7efdff [term 1 FOLLOWER]: Refusing update from remote peer d0120f0908884dcdad627e94725c4823: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:23:02.391340 24796 raft_consensus.cc:1261] T 854ecd93e67f49f6b8e8f9e16e4a06be P 0e588bd328bd493797faf8b912f8b24b [term 1 FOLLOWER]: Refusing update from remote peer d0120f0908884dcdad627e94725c4823: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:23:02.391680 24910 consensus_queue.cc:1035] T 854ecd93e67f49f6b8e8f9e16e4a06be P d0120f0908884dcdad627e94725c4823 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e588bd328bd493797faf8b912f8b24b" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 38539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:23:02.391788 24908 consensus_queue.cc:1035] T 854ecd93e67f49f6b8e8f9e16e4a06be P d0120f0908884dcdad627e94725c4823 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06b7eba5a9434d7f97de86825b7efdff" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 35807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:23:02.394706 24915 mvcc.cc:205] Tried to move back new op lower bound from 7317494916672278528 to 7317494916639457280. Current Snapshot: MvccSnapshot[applied={T|T < 7317494916672278528}]
I20260812 01:23:02.394981 24914 mvcc.cc:205] Tried to move back new op lower bound from 7317494916672278528 to 7317494916639457280. Current Snapshot: MvccSnapshot[applied={T|T < 7317494916672278528}]
I20260812 01:23:02.396461 24121 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53106:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:23:02.396572 24121 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53106:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:23:02.397356 24121 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9a9cf8116f71488c99dad819f55bf5b8: Sending DeleteTablet for 3 replicas of tablet 854ecd93e67f49f6b8e8f9e16e4a06be
I20260812 01:23:02.397706 24308 tablet_service.cc:1514] Processing DeleteTablet for tablet 854ecd93e67f49f6b8e8f9e16e4a06be with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:23:02 UTC) from {username='kudu', principal='kudu/127.24.0.190@KRBTEST.COM'} at 127.0.0.1:36850
I20260812 01:23:02.397953 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 24211
I20260812 01:23:02.397967 24935 tablet_replica.cc:330] T 854ecd93e67f49f6b8e8f9e16e4a06be P d0120f0908884dcdad627e94725c4823: stopping tablet replica
I20260812 01:23:02.397951 24517 tablet_service.cc:1514] Processing DeleteTablet for tablet 854ecd93e67f49f6b8e8f9e16e4a06be with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:23:02 UTC) from {username='kudu', principal='kudu/127.24.0.190@KRBTEST.COM'} at 127.0.0.1:54202
I20260812 01:23:02.398074 24935 raft_consensus.cc:2229] T 854ecd93e67f49f6b8e8f9e16e4a06be P d0120f0908884dcdad627e94725c4823 [term 1 LEADER]: Raft consensus shutting down.
I20260812 01:23:02.398161 24935 raft_consensus.cc:2258] T 854ecd93e67f49f6b8e8f9e16e4a06be P d0120f0908884dcdad627e94725c4823 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:23:02.402902 24766 tablet_service.cc:1514] Processing DeleteTablet for tablet 854ecd93e67f49f6b8e8f9e16e4a06be with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:23:02 UTC) from {username='kudu', principal='kudu/127.24.0.190@KRBTEST.COM'} at 127.0.0.1:45258
I20260812 01:23:02.404170 24940 tablet_replica.cc:330] T 854ecd93e67f49f6b8e8f9e16e4a06be P 0e588bd328bd493797faf8b912f8b24b: stopping tablet replica
I20260812 01:23:02.404273 24940 raft_consensus.cc:2229] T 854ecd93e67f49f6b8e8f9e16e4a06be P 0e588bd328bd493797faf8b912f8b24b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 01:23:02.404376 24940 raft_consensus.cc:2258] T 854ecd93e67f49f6b8e8f9e16e4a06be P 0e588bd328bd493797faf8b912f8b24b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:23:02.406309 24940 ts_tablet_manager.cc:1905] T 854ecd93e67f49f6b8e8f9e16e4a06be P 0e588bd328bd493797faf8b912f8b24b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:23:02.407135 24937 tablet_replica.cc:330] T 854ecd93e67f49f6b8e8f9e16e4a06be P 06b7eba5a9434d7f97de86825b7efdff: stopping tablet replica
I20260812 01:23:02.407341 24937 raft_consensus.cc:2229] T 854ecd93e67f49f6b8e8f9e16e4a06be P 06b7eba5a9434d7f97de86825b7efdff [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 01:23:02.407572 24937 raft_consensus.cc:2258] T 854ecd93e67f49f6b8e8f9e16e4a06be P 06b7eba5a9434d7f97de86825b7efdff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:23:02.407819 24940 ts_tablet_manager.cc:1918] T 854ecd93e67f49f6b8e8f9e16e4a06be P 0e588bd328bd493797faf8b912f8b24b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 01:23:02.407939 24940 log.cc:1196] T 854ecd93e67f49f6b8e8f9e16e4a06be P 0e588bd328bd493797faf8b912f8b24b: Deleting WAL directory at /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786497534195564-24578-0/minicluster-data/ts-2/wal/wals/854ecd93e67f49f6b8e8f9e16e4a06be
I20260812 01:23:02.408304 24940 ts_tablet_manager.cc:1939] T 854ecd93e67f49f6b8e8f9e16e4a06be P 0e588bd328bd493797faf8b912f8b24b: Deleting consensus metadata
I20260812 01:23:02.408756 24098 catalog_manager.cc:4841] TS 0e588bd328bd493797faf8b912f8b24b (127.24.0.131:38539): tablet 854ecd93e67f49f6b8e8f9e16e4a06be (table test-table [id=e5f84ae72a4e47efb3ed0ef7095aaab4]) successfully deleted
I20260812 01:23:02.409337 24937 ts_tablet_manager.cc:1905] T 854ecd93e67f49f6b8e8f9e16e4a06be P 06b7eba5a9434d7f97de86825b7efdff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:23:02.411688 24937 ts_tablet_manager.cc:1918] T 854ecd93e67f49f6b8e8f9e16e4a06be P 06b7eba5a9434d7f97de86825b7efdff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 01:23:02.411796 24937 log.cc:1196] T 854ecd93e67f49f6b8e8f9e16e4a06be P 06b7eba5a9434d7f97de86825b7efdff: Deleting WAL directory at /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786497534195564-24578-0/minicluster-data/ts-1/wal/wals/854ecd93e67f49f6b8e8f9e16e4a06be
I20260812 01:23:02.412274 24937 ts_tablet_manager.cc:1939] T 854ecd93e67f49f6b8e8f9e16e4a06be P 06b7eba5a9434d7f97de86825b7efdff: Deleting consensus metadata
I20260812 01:23:02.412741 24095 catalog_manager.cc:4841] TS 06b7eba5a9434d7f97de86825b7efdff (127.24.0.130:35807): tablet 854ecd93e67f49f6b8e8f9e16e4a06be (table test-table [id=e5f84ae72a4e47efb3ed0ef7095aaab4]) successfully deleted
W20260812 01:23:02.418601 24100 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.0.129:34935 (error 108)
I20260812 01:23:02.419195 24889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:02.419057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.129:34935 (local address 127.0.0.1:36862)
0812 01:23:02.419122 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0812 01:23:02.419165 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.129:34935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260812 01:23:02.419345 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 24417
W20260812 01:23:02.419356 24100 catalog_manager.cc:4584] TS d0120f0908884dcdad627e94725c4823 (127.24.0.129:34935): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 854ecd93e67f49f6b8e8f9e16e4a06be: Network error: Client connection negotiation failed: client connection to 127.24.0.129:34935: connect: Connection refused (error 111)
I20260812 01:23:02.429611 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 24652
I20260812 01:23:02.443562 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 24060
2026-08-12T01:23:02Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2208 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-12T01:23:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:23:02Z Disabled control of system clock
I20260812 01:23:02.477041 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786497534195564-24578-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786497534195564-24578-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786497534195564-24578-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.190:41079
--webserver_interface=127.24.0.190
--webserver_port=0
--builtin_ntp_servers=127.24.0.148:45125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.0.190:41079
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 01:23:02.560583 24946 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:23:02.560808 24946 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:23:02.560830 24946 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:23:02.562003 24946 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:23:02.562049 24946 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:23:02.562070 24946 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:23:02.562122 24946 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:23:02.562150 24946 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:23:02.563494 24946 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:45125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786497534195564-24578-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786497534195564-24578-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.0.190:41079
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.190:41079
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.0.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24946
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786497534195564-24578-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:23:02.563752 24946 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:23:02.564085 24946 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:23:02.569001 24956 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:23:02.569078 24954 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:23:02.569000 24953 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:23:02.569242 24946 server_base.cc:1034] running on GCE node
I20260812 01:23:02.569525 24946 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:23:02.569873 24946 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:23:02.571071 24946 hybrid_clock.cc:648] HybridClock initialized: now 1786497782571064 us; error 40 us; skew 500 ppm
I20260812 01:23:02.572556 24946 webserver.cc:460] Webserver started at http://127.24.0.190:45383/ using document root <none> and password file <none>
I20260812 01:23:02.572820 24946 fs_manager.cc:359] Metadata directory not provided
I20260812 01:23:02.572892 24946 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:23:02.573089 24946 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:23:02.574220 24946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786497534195564-24578-0/minicluster-data/master-0/data/instance:
uuid: "8eb1c87ac6074213b950e85bd694b59f"
format_stamp: "Formatted at 2026-08-12 01:23:02 on dist-test-slave-t3q3"
server_key: "48e27e82a28729554893518c2965abb7"
server_key_iv: "42a8c0849dc7aaa682bd2264fbb81f26"
server_key_version: "encryptionkey@0"
I20260812 01:23:02.574606 24946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786497534195564-24578-0/minicluster-data/master-0/wal/instance:
uuid: "8eb1c87ac6074213b950e85bd694b59f"
format_stamp: "Formatted at 2026-08-12 01:23:02 on dist-test-slave-t3q3"
server_key: "48e27e82a28729554893518c2965abb7"
server_key_iv: "42a8c0849dc7aaa682bd2264fbb81f26"
server_key_version: "encryptionkey@0"
I20260812 01:23:02.576006 24946 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260812 01:23:02.576880 24965 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:23:02.577077 24946 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 01:23:02.577183 24946 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786497534195564-24578-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786497534195564-24578-0/minicluster-data/master-0/wal
uuid: "8eb1c87ac6074213b950e85bd694b59f"
format_stamp: "Formatted at 2026-08-12 01:23:02 on dist-test-slave-t3q3"
server_key: "48e27e82a28729554893518c2965abb7"
server_key_iv: "42a8c0849dc7aaa682bd2264fbb81f26"
server_key_version: "encryptionkey@0"
I20260812 01:23:02.577266 24946 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786497534195564-24578-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786497534195564-24578-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786497534195564-24578-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:23:02.591697 24946 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:23:02.592005 24946 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:23:02.592197 24946 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:23:02.596331 24946 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.190:41079
I20260812 01:23:02.596376 25052 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.190:41079 every 8 connection(s)
I20260812 01:23:02.596709 24946 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
I20260812 01:23:02.597635 25053 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:23:02.599941 25053 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8eb1c87ac6074213b950e85bd694b59f: Bootstrap starting.
I20260812 01:23:02.600597 25053 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8eb1c87ac6074213b950e85bd694b59f: Neither blocks nor log segments found. Creating new log.
I20260812 01:23:02.600917 25053 log.cc:824] T 00000000000000000000000000000000 P 8eb1c87ac6074213b950e85bd694b59f: Log is configured to *not* fsync() on all Append() calls
I20260812 01:23:02.601651 25053 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8eb1c87ac6074213b950e85bd694b59f: No bootstrap required, opened a new log
I20260812 01:23:02.601716 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 24946
I20260812 01:23:02.601855 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786497534195564-24578-0/minicluster-data/master-0/wal/instance
I20260812 01:23:02.602092 24578 external_mini_cluster.cc:1428] Setting key 62c854a888ad037f62b97ba6034f819d
I20260812 01:23:02.603726 25053 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8eb1c87ac6074213b950e85bd694b59f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8eb1c87ac6074213b950e85bd694b59f" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 41079 } }
I20260812 01:23:02.603890 25053 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8eb1c87ac6074213b950e85bd694b59f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:23:02.603940 25053 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8eb1c87ac6074213b950e85bd694b59f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8eb1c87ac6074213b950e85bd694b59f, State: Initialized, Role: FOLLOWER
I20260812 01:23:02.604133 25053 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8eb1c87ac6074213b950e85bd694b59f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8eb1c87ac6074213b950e85bd694b59f" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 41079 } }
I20260812 01:23:02.604239 25053 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8eb1c87ac6074213b950e85bd694b59f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:23:02.604280 25053 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8eb1c87ac6074213b950e85bd694b59f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:23:02.604334 25053 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8eb1c87ac6074213b950e85bd694b59f [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:23:02.604894 25053 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8eb1c87ac6074213b950e85bd694b59f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8eb1c87ac6074213b950e85bd694b59f" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 41079 } }
I20260812 01:23:02.604982 25053 leader_election.cc:302] T 00000000000000000000000000000000 P 8eb1c87ac6074213b950e85bd694b59f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8eb1c87ac6074213b950e85bd694b59f; no voters:
I20260812 01:23:02.605219 25053 leader_election.cc:288] T 00000000000000000000000000000000 P 8eb1c87ac6074213b950e85bd694b59f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 01:23:02.605250 25059 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8eb1c87ac6074213b950e85bd694b59f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:23:02.605369 25059 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8eb1c87ac6074213b950e85bd694b59f [term 1 LEADER]: Becoming Leader. State: Replica: 8eb1c87ac6074213b950e85bd694b59f, State: Running, Role: LEADER
I20260812 01:23:02.605512 25059 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8eb1c87ac6074213b950e85bd694b59f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8eb1c87ac6074213b950e85bd694b59f" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 41079 } }
I20260812 01:23:02.605517 25053 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8eb1c87ac6074213b950e85bd694b59f [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:23:02.606102 25061 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8eb1c87ac6074213b950e85bd694b59f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8eb1c87ac6074213b950e85bd694b59f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8eb1c87ac6074213b950e85bd694b59f" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 41079 } } }
I20260812 01:23:02.606122 25060 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8eb1c87ac6074213b950e85bd694b59f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8eb1c87ac6074213b950e85bd694b59f. Latest consensus state: current_term: 1 leader_uuid: "8eb1c87ac6074213b950e85bd694b59f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8eb1c87ac6074213b950e85bd694b59f" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 41079 } } }
I20260812 01:23:02.606210 25061 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8eb1c87ac6074213b950e85bd694b59f [sys.catalog]: This master's current role is: LEADER
I20260812 01:23:02.606235 25060 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8eb1c87ac6074213b950e85bd694b59f [sys.catalog]: This master's current role is: LEADER
I20260812 01:23:02.606508 25067 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:23:02.607024 25067 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:23:02.608875 25067 catalog_manager.cc:1347] Generated new cluster ID: 97086f5e01d1471ea151beda6a1b23f3
I20260812 01:23:02.608937 25067 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 01:23:02.609602 25057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:02.602684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45950 (local address 127.24.0.190:41079)
0812 01:23:02.602891 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:02.602898 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:02.602922 (+ 24us) server_negotiation.cc:407] Connection header received
0812 01:23:02.603747 (+ 825us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:02.603756 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:02.603831 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:02.604014 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:23:02.604310 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:02.606125 (+ 1815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:02.607452 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:02.607906 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:02.608095 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:02.608752 (+ 657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:23:02.608756 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:23:02.608756 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:23:02.609026 (+ 270us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:23:02.609074 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:23:02.609097 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:23:02.609181 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:02.609317 (+ 136us) server_negotiation.cc:299] Negotiation successful
0812 01:23:02.609409 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":91,"threads_started":1}
I20260812 01:23:02.619095 25067 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:23:02.619637 25067 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:23:02.625219 25067 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8eb1c87ac6074213b950e85bd694b59f: Generated new TSK 0
I20260812 01:23:02.625418 25067 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 01:23:02.631016 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786497534195564-24578-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786497534195564-24578-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786497534195564-24578-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786497534195564-24578-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.0.129:0
--local_ip_for_outbound_sockets=127.24.0.129
--webserver_interface=127.24.0.129
--webserver_port=0
--tserver_master_addrs=127.24.0.190:41079
--builtin_ntp_servers=127.24.0.148:45125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 01:23:02.715689 25083 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:23:02.715899 25083 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:23:02.715922 25083 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:23:02.717175 25083 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:23:02.717217 25083 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:23:02.717242 25083 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.0.129
I20260812 01:23:02.718585 25083 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:45125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786497534195564-24578-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786497534195564-24578-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.0.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786497534195564-24578-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.0.129
--webserver_port=0
--tserver_master_addrs=127.24.0.190:41079
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25083
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.0.129
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786497534195564-24578-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:23:02.718859 25083 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:23:02.719177 25083 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:23:02.725304 25092 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:23:02.725296 25093 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:23:02.725296 25095 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:23:02.725839 25083 server_base.cc:1034] running on GCE node
I20260812 01:23:02.726058 25083 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:23:02.726387 25083 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:23:02.727566 25083 hybrid_clock.cc:648] HybridClock initialized: now 1786497782727548 us; error 39 us; skew 500 ppm
I20260812 01:23:02.729156 25083 webserver.cc:460] Webserver started at http://127.24.0.129:37227/ using document root <none> and password file <none>
I20260812 01:23:02.729483 25083 fs_manager.cc:359] Metadata directory not provided
I20260812 01:23:02.729574 25083 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:23:02.729730 25083 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:23:02.731510 25083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786497534195564-24578-0/minicluster-data/ts-0/data/instance:
uuid: "6532f4e6bbe74b9eb0a6988d2ca9f943"
format_stamp: "Formatted at 2026-08-12 01:23:02 on dist-test-slave-t3q3"
server_key: "64d136c9e29bf85c6701bb2be6a5b02f"
server_key_iv: "6d695be58e08472df2a97213dcd6b310"
server_key_version: "encryptionkey@0"
I20260812 01:23:02.732009 25083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786497534195564-24578-0/minicluster-data/ts-0/wal/instance:
uuid: "6532f4e6bbe74b9eb0a6988d2ca9f943"
format_stamp: "Formatted at 2026-08-12 01:23:02 on dist-test-slave-t3q3"
server_key: "64d136c9e29bf85c6701bb2be6a5b02f"
server_key_iv: "6d695be58e08472df2a97213dcd6b310"
server_key_version: "encryptionkey@0"
I20260812 01:23:02.734418 25083 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.001s
I20260812 01:23:02.735859 25104 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:23:02.736034 25083 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:23:02.736202 25083 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786497534195564-24578-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786497534195564-24578-0/minicluster-data/ts-0/wal
uuid: "6532f4e6bbe74b9eb0a6988d2ca9f943"
format_stamp: "Formatted at 2026-08-12 01:23:02 on dist-test-slave-t3q3"
server_key: "64d136c9e29bf85c6701bb2be6a5b02f"
server_key_iv: "6d695be58e08472df2a97213dcd6b310"
server_key_version: "encryptionkey@0"
I20260812 01:23:02.736301 25083 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786497534195564-24578-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786497534195564-24578-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786497534195564-24578-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:23:02.757272 25083 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:23:02.757640 25083 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:23:02.757805 25083 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:23:02.758072 25083 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:23:02.758462 25083 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:23:02.758522 25083 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:23:02.758566 25083 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:23:02.758594 25083 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:23:02.764688 25083 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.129:34411
I20260812 01:23:02.764744 25273 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.129:34411 every 8 connection(s)
I20260812 01:23:02.765153 25083 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786497534195564-24578-0/minicluster-data/ts-0/data/info.pb
I20260812 01:23:02.767545 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 25083
I20260812 01:23:02.767665 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786497534195564-24578-0/minicluster-data/ts-0/wal/instance
I20260812 01:23:02.767879 24578 external_mini_cluster.cc:1428] Setting key 4efb1ce3c8b1d2764d2b9101cc8f9a05
I20260812 01:23:02.768872 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786497534195564-24578-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786497534195564-24578-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786497534195564-24578-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786497534195564-24578-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.0.130:0
--local_ip_for_outbound_sockets=127.24.0.130
--webserver_interface=127.24.0.130
--webserver_port=0
--tserver_master_addrs=127.24.0.190:41079
--builtin_ntp_servers=127.24.0.148:45125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 01:23:02.770793 25057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:02.765979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.129:46713 (local address 127.24.0.190:41079)
0812 01:23:02.766069 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:02.766071 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:02.766296 (+ 225us) server_negotiation.cc:407] Connection header received
0812 01:23:02.766782 (+ 486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:02.766783 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:02.766796 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:02.766851 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:23:02.767667 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:02.768272 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:02.769293 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:02.769490 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:02.769568 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:02.770476 (+ 908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:23:02.770478 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:23:02.770478 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:23:02.770483 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:23:02.770506 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:23:02.770511 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:23:02.770569 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:02.770679 (+ 110us) server_negotiation.cc:299] Negotiation successful
0812 01:23:02.770711 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 01:23:02.771042 25279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:02.765938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:41079 (local address 127.24.0.129:46713)
0812 01:23:02.766170 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0812 01:23:02.766206 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 01:23:02.766700 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:23:02.766907 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:23:02.766910 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:23:02.766917 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:23:02.767563 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:23:02.767571 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:23:02.768379 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:23:02.768380 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:23:02.769182 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:23:02.769184 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:23:02.769580 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:23:02.769582 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:23:02.769720 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:02.770327 (+ 607us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 01:23:02.770327 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:23:02.770358 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 01:23:02.770383 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:23:02.770585 (+ 202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:23:02.770598 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:23:02.770602 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 01:23:02.770681 (+ 79us) client_negotiation.cc:240] Negotiation successful
0812 01:23:02.770791 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":120,"threads_started":1}
I20260812 01:23:02.771585 25274 heartbeater.cc:347] Connected to a master server at 127.24.0.190:41079
I20260812 01:23:02.771705 25274 heartbeater.cc:464] Registering TS with master...
I20260812 01:23:02.771927 25274 heartbeater.cc:511] Master 127.24.0.190:41079 requested a full tablet report, sending...
I20260812 01:23:02.772492 24995 ts_manager.cc:194] Registered new tserver with Master: 6532f4e6bbe74b9eb0a6988d2ca9f943 (127.24.0.129:34411)
I20260812 01:23:02.773243 24995 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.0.129:46713
W20260812 01:23:02.853652 25281 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:23:02.853868 25281 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:23:02.853890 25281 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:23:02.855051 25281 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:23:02.855094 25281 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:23:02.855119 25281 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.0.130
I20260812 01:23:02.856537 25281 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:45125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786497534195564-24578-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786497534195564-24578-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.0.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786497534195564-24578-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.0.130
--webserver_port=0
--tserver_master_addrs=127.24.0.190:41079
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25281
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.0.130
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786497534195564-24578-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:23:02.856804 25281 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:23:02.857127 25281 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:23:02.862191 25289 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:23:02.862342 25281 server_base.cc:1034] running on GCE node
W20260812 01:23:02.862190 25293 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:23:02.862200 25288 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:23:02.862628 25281 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:23:02.862900 25281 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:23:02.864135 25281 hybrid_clock.cc:648] HybridClock initialized: now 1786497782864127 us; error 39 us; skew 500 ppm
I20260812 01:23:02.865442 25281 webserver.cc:460] Webserver started at http://127.24.0.130:44779/ using document root <none> and password file <none>
I20260812 01:23:02.865724 25281 fs_manager.cc:359] Metadata directory not provided
I20260812 01:23:02.865798 25281 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:23:02.865955 25281 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:23:02.867048 25281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786497534195564-24578-0/minicluster-data/ts-1/data/instance:
uuid: "6d117a85b2ff4b4186f3edabd6047f43"
format_stamp: "Formatted at 2026-08-12 01:23:02 on dist-test-slave-t3q3"
server_key: "83b733e2956f4c9f59b8dd557dff032d"
server_key_iv: "13bab60261f33c98753beea6b8dd4ae0"
server_key_version: "encryptionkey@0"
I20260812 01:23:02.867431 25281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786497534195564-24578-0/minicluster-data/ts-1/wal/instance:
uuid: "6d117a85b2ff4b4186f3edabd6047f43"
format_stamp: "Formatted at 2026-08-12 01:23:02 on dist-test-slave-t3q3"
server_key: "83b733e2956f4c9f59b8dd557dff032d"
server_key_iv: "13bab60261f33c98753beea6b8dd4ae0"
server_key_version: "encryptionkey@0"
I20260812 01:23:02.868863 25281 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 01:23:02.869637 25307 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:23:02.869820 25281 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 01:23:02.869930 25281 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786497534195564-24578-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786497534195564-24578-0/minicluster-data/ts-1/wal
uuid: "6d117a85b2ff4b4186f3edabd6047f43"
format_stamp: "Formatted at 2026-08-12 01:23:02 on dist-test-slave-t3q3"
server_key: "83b733e2956f4c9f59b8dd557dff032d"
server_key_iv: "13bab60261f33c98753beea6b8dd4ae0"
server_key_version: "encryptionkey@0"
I20260812 01:23:02.870009 25281 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786497534195564-24578-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786497534195564-24578-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786497534195564-24578-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:23:02.886441 25281 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:23:02.886780 25281 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:23:02.886947 25281 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:23:02.887224 25281 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:23:02.887612 25281 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:23:02.887673 25281 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:23:02.887717 25281 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:23:02.887742 25281 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:23:02.893985 25281 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.130:41843
I20260812 01:23:02.894037 25484 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.130:41843 every 8 connection(s)
I20260812 01:23:02.894394 25281 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786497534195564-24578-0/minicluster-data/ts-1/data/info.pb
I20260812 01:23:02.895725 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 25281
I20260812 01:23:02.895869 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786497534195564-24578-0/minicluster-data/ts-1/wal/instance
I20260812 01:23:02.896137 24578 external_mini_cluster.cc:1428] Setting key a99d19c8bf4566b57392f77f57d52907
I20260812 01:23:02.897282 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786497534195564-24578-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786497534195564-24578-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786497534195564-24578-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786497534195564-24578-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.0.131:0
--local_ip_for_outbound_sockets=127.24.0.131
--webserver_interface=127.24.0.131
--webserver_port=0
--tserver_master_addrs=127.24.0.190:41079
--builtin_ntp_servers=127.24.0.148:45125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 01:23:02.900640 25057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:02.895230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.130:45281 (local address 127.24.0.190:41079)
0812 01:23:02.895312 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:02.895314 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:02.895608 (+ 294us) server_negotiation.cc:407] Connection header received
0812 01:23:02.896284 (+ 676us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:02.896286 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:02.896302 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:02.896384 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:23:02.897247 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:02.898053 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:02.899055 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:02.899280 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:02.899364 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:02.900331 (+ 967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:23:02.900332 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:23:02.900332 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:23:02.900338 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:23:02.900354 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:23:02.900358 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:23:02.900415 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:02.900526 (+ 111us) server_negotiation.cc:299] Negotiation successful
0812 01:23:02.900557 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 01:23:02.900925 25487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:02.895232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:41079 (local address 127.24.0.130:45281)
0812 01:23:02.895469 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0812 01:23:02.895505 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 01:23:02.896117 (+ 612us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:23:02.896485 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:23:02.896489 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:23:02.896500 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:23:02.897140 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:23:02.897146 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:23:02.898156 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:23:02.898157 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:23:02.898953 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:23:02.898955 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:23:02.899373 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:23:02.899374 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:23:02.899520 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:02.900159 (+ 639us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 01:23:02.900159 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:23:02.900192 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 01:23:02.900217 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:23:02.900425 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:23:02.900430 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:23:02.900433 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 01:23:02.900518 (+ 85us) client_negotiation.cc:240] Negotiation successful
0812 01:23:02.900638 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":110,"threads_started":1}
I20260812 01:23:02.901407 25485 heartbeater.cc:347] Connected to a master server at 127.24.0.190:41079
I20260812 01:23:02.901530 25485 heartbeater.cc:464] Registering TS with master...
I20260812 01:23:02.901762 25485 heartbeater.cc:511] Master 127.24.0.190:41079 requested a full tablet report, sending...
I20260812 01:23:02.902221 24995 ts_manager.cc:194] Registered new tserver with Master: 6d117a85b2ff4b4186f3edabd6047f43 (127.24.0.130:41843)
I20260812 01:23:02.902603 24995 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.0.130:45281
W20260812 01:23:02.992249 25489 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:23:02.992479 25489 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:23:02.992511 25489 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:23:02.993876 25489 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:23:02.993922 25489 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:23:02.993952 25489 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.0.131
I20260812 01:23:02.995446 25489 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:45125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786497534195564-24578-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786497534195564-24578-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.0.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786497534195564-24578-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.0.131
--webserver_port=0
--tserver_master_addrs=127.24.0.190:41079
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25489
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.0.131
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786497534195564-24578-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:23:02.995671 25489 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:23:02.995967 25489 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:23:03.001433 25500 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:23:03.001513 25499 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:23:03.001433 25503 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:23:03.001706 25489 server_base.cc:1034] running on GCE node
I20260812 01:23:03.001960 25489 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:23:03.002183 25489 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:23:03.003360 25489 hybrid_clock.cc:648] HybridClock initialized: now 1786497783003351 us; error 42 us; skew 500 ppm
I20260812 01:23:03.004717 25489 webserver.cc:460] Webserver started at http://127.24.0.131:44637/ using document root <none> and password file <none>
I20260812 01:23:03.005105 25489 fs_manager.cc:359] Metadata directory not provided
I20260812 01:23:03.005208 25489 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:23:03.005440 25489 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:23:03.006783 25489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786497534195564-24578-0/minicluster-data/ts-2/data/instance:
uuid: "446790d853a34e6db0de8d5ffa39e90f"
format_stamp: "Formatted at 2026-08-12 01:23:03 on dist-test-slave-t3q3"
server_key: "0a319a033dc414e8f048d5d79788e3b8"
server_key_iv: "732b0cb823dba6853e9be7775d374cac"
server_key_version: "encryptionkey@0"
I20260812 01:23:03.007167 25489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786497534195564-24578-0/minicluster-data/ts-2/wal/instance:
uuid: "446790d853a34e6db0de8d5ffa39e90f"
format_stamp: "Formatted at 2026-08-12 01:23:03 on dist-test-slave-t3q3"
server_key: "0a319a033dc414e8f048d5d79788e3b8"
server_key_iv: "732b0cb823dba6853e9be7775d374cac"
server_key_version: "encryptionkey@0"
I20260812 01:23:03.008649 25489 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 01:23:03.009471 25512 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:23:03.009684 25489 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 01:23:03.009789 25489 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786497534195564-24578-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786497534195564-24578-0/minicluster-data/ts-2/wal
uuid: "446790d853a34e6db0de8d5ffa39e90f"
format_stamp: "Formatted at 2026-08-12 01:23:03 on dist-test-slave-t3q3"
server_key: "0a319a033dc414e8f048d5d79788e3b8"
server_key_iv: "732b0cb823dba6853e9be7775d374cac"
server_key_version: "encryptionkey@0"
I20260812 01:23:03.009862 25489 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786497534195564-24578-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786497534195564-24578-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786497534195564-24578-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:23:03.032272 25489 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:23:03.032661 25489 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:23:03.032836 25489 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:23:03.033145 25489 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:23:03.033519 25489 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:23:03.033586 25489 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:23:03.033623 25489 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:23:03.033646 25489 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:23:03.039494 25489 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.131:44621
I20260812 01:23:03.039541 25679 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.131:44621 every 8 connection(s)
I20260812 01:23:03.039893 25489 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786497534195564-24578-0/minicluster-data/ts-2/data/info.pb
I20260812 01:23:03.044670 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 25489
I20260812 01:23:03.044811 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786497534195564-24578-0/minicluster-data/ts-2/wal/instance
I20260812 01:23:03.045050 24578 external_mini_cluster.cc:1428] Setting key 201bb02917ee3ec2da62fffdbda2c992
I20260812 01:23:03.045212 25057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:03.040759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.131:45583 (local address 127.24.0.190:41079)
0812 01:23:03.040830 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:03.040831 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:03.041109 (+ 278us) server_negotiation.cc:407] Connection header received
0812 01:23:03.041568 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:03.041569 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:03.041579 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:03.041639 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:23:03.042295 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:03.042778 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:03.043736 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:03.043917 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:03.043972 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:03.044906 (+ 934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:23:03.044908 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:23:03.044908 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:23:03.044913 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:23:03.044932 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:23:03.044938 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:23:03.045007 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:03.045082 (+ 75us) server_negotiation.cc:299] Negotiation successful
0812 01:23:03.045123 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 01:23:03.045481 25682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:03.040725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:41079 (local address 127.24.0.131:45583)
0812 01:23:03.040957 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0812 01:23:03.040999 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 01:23:03.041485 (+ 486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:23:03.041701 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:23:03.041705 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:23:03.041713 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:23:03.042227 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:23:03.042231 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:23:03.042862 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:23:03.042864 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:23:03.043650 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:23:03.043652 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:23:03.043998 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:23:03.043999 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:23:03.044129 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:03.044726 (+ 597us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 01:23:03.044727 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:23:03.044757 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 01:23:03.044780 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:23:03.045008 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:23:03.045013 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:23:03.045015 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 01:23:03.045072 (+ 57us) client_negotiation.cc:240] Negotiation successful
0812 01:23:03.045213 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":129,"threads_started":1}
I20260812 01:23:03.046146 25680 heartbeater.cc:347] Connected to a master server at 127.24.0.190:41079
I20260812 01:23:03.046262 25680 heartbeater.cc:464] Registering TS with master...
I20260812 01:23:03.046448 25680 heartbeater.cc:511] Master 127.24.0.190:41079 requested a full tablet report, sending...
I20260812 01:23:03.046845 24993 ts_manager.cc:194] Registered new tserver with Master: 446790d853a34e6db0de8d5ffa39e90f (127.24.0.131:44621)
I20260812 01:23:03.047219 24993 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.0.131:45583
I20260812 01:23:03.057590 24578 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 01:23:03.061717 25057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:03.059068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45952 (local address 127.24.0.190:41079)
0812 01:23:03.059136 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:03.059138 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:03.059257 (+ 119us) server_negotiation.cc:407] Connection header received
0812 01:23:03.059297 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:03.059297 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:03.059309 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:03.059362 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:23:03.059507 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:03.059967 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:03.060706 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:03.060883 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:03.060937 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:03.061503 (+ 566us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:23:03.061504 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:23:03.061505 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:23:03.061508 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:23:03.061521 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:23:03.061525 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:23:03.061568 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:03.061640 (+ 72us) server_negotiation.cc:299] Negotiation successful
0812 01:23:03.061661 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 01:23:03.063321 24993 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45952:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 01:23:03.064824 24993 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 01:23:03.071183 25694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:03.067127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32786 (local address 127.24.0.129:34411)
0812 01:23:03.067448 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:03.067454 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:03.067469 (+ 15us) server_negotiation.cc:407] Connection header received
0812 01:23:03.067528 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:03.067531 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:03.067547 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:03.067822 (+ 275us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:23:03.067890 (+ 68us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:03.069035 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:03.070264 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:03.070764 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:03.070859 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:03.070958 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:03.071054 (+ 96us) server_negotiation.cc:299] Negotiation successful
0812 01:23:03.071091 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":272,"thread_start_us":105,"threads_started":1}
I20260812 01:23:03.071293 25693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:03.067230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.131:44621 (local address 127.0.0.1:52240)
0812 01:23:03.067461 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0812 01:23:03.067471 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 01:23:03.067514 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:23:03.067856 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:23:03.067857 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:23:03.067859 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:23:03.067894 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:23:03.067895 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:23:03.069157 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:23:03.069159 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:23:03.070063 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:23:03.070065 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:23:03.071079 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:23:03.071080 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:23:03.071143 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:03.071146 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 01:23:03.071196 (+ 50us) client_negotiation.cc:240] Negotiation successful
0812 01:23:03.071236 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":87,"threads_started":1}
I20260812 01:23:03.071488 25699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:03.067228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57804 (local address 127.24.0.130:41843)
0812 01:23:03.067591 (+ 363us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:03.067596 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:03.067608 (+ 12us) server_negotiation.cc:407] Connection header received
0812 01:23:03.067652 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:03.067654 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:03.067670 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:03.067919 (+ 249us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:23:03.067978 (+ 59us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:03.068789 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:03.070206 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:03.070887 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:03.071250 (+ 363us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:03.071347 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:03.071375 (+ 28us) server_negotiation.cc:299] Negotiation successful
0812 01:23:03.071415 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":91,"threads_started":1}
I20260812 01:23:03.071519 25700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:03.067440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52240 (local address 127.24.0.131:44621)
0812 01:23:03.067664 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:03.067668 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:03.067677 (+ 9us) server_negotiation.cc:407] Connection header received
0812 01:23:03.067708 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:03.067710 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:03.067722 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:03.067796 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:23:03.067986 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:03.069069 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:03.070161 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:03.070887 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:03.070962 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:03.071047 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:03.071401 (+ 354us) server_negotiation.cc:299] Negotiation successful
0812 01:23:03.071439 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":89,"threads_started":1}
I20260812 01:23:03.071717 25692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:03.067012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.130:41843 (local address 127.0.0.1:57804)
0812 01:23:03.067193 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0812 01:23:03.067230 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 01:23:03.067294 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:23:03.067786 (+ 492us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:23:03.067787 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:23:03.067789 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:23:03.067839 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:23:03.067840 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:23:03.068874 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:23:03.068875 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:23:03.070127 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:23:03.070129 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:23:03.070994 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:23:03.070995 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:23:03.071058 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:03.071062 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 01:23:03.071095 (+ 33us) client_negotiation.cc:240] Negotiation successful
0812 01:23:03.071134 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":65,"threads_started":1}
I20260812 01:23:03.071204 25695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:03.067012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.129:34411 (local address 127.0.0.1:32786)
0812 01:23:03.067373 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0812 01:23:03.067384 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 01:23:03.067444 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:23:03.067666 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:23:03.067669 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:23:03.067671 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:23:03.067737 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:23:03.067739 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:23:03.069129 (+ 1390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:23:03.069130 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:23:03.070177 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:23:03.070179 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:23:03.070860 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:23:03.070861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:23:03.070937 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:03.070950 (+ 13us) client_negotiation.cc:769] Sending connection context
0812 01:23:03.071031 (+ 81us) client_negotiation.cc:240] Negotiation successful
0812 01:23:03.071090 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":341,"mutex_wait_us":2,"thread_start_us":82,"threads_started":1}
I20260812 01:23:03.072223 25590 tablet_service.cc:1467] Processing CreateTablet for tablet 4424797293c14909b54a4c21c02ff5ec (DEFAULT_TABLE table=test-table [id=e66f6f2c90654d2c8f48e8ab098ae829]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:23:03.072456 25389 tablet_service.cc:1467] Processing CreateTablet for tablet 4424797293c14909b54a4c21c02ff5ec (DEFAULT_TABLE table=test-table [id=e66f6f2c90654d2c8f48e8ab098ae829]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:23:03.072456 25178 tablet_service.cc:1467] Processing CreateTablet for tablet 4424797293c14909b54a4c21c02ff5ec (DEFAULT_TABLE table=test-table [id=e66f6f2c90654d2c8f48e8ab098ae829]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:23:03.072669 25590 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4424797293c14909b54a4c21c02ff5ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:23:03.072814 25389 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4424797293c14909b54a4c21c02ff5ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:23:03.072831 25178 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4424797293c14909b54a4c21c02ff5ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:23:03.075541 25702 tablet_bootstrap.cc:492] T 4424797293c14909b54a4c21c02ff5ec P 446790d853a34e6db0de8d5ffa39e90f: Bootstrap starting.
I20260812 01:23:03.076179 25703 tablet_bootstrap.cc:492] T 4424797293c14909b54a4c21c02ff5ec P 6532f4e6bbe74b9eb0a6988d2ca9f943: Bootstrap starting.
I20260812 01:23:03.076349 25702 tablet_bootstrap.cc:654] T 4424797293c14909b54a4c21c02ff5ec P 446790d853a34e6db0de8d5ffa39e90f: Neither blocks nor log segments found. Creating new log.
I20260812 01:23:03.076704 25702 log.cc:824] T 4424797293c14909b54a4c21c02ff5ec P 446790d853a34e6db0de8d5ffa39e90f: Log is configured to *not* fsync() on all Append() calls
I20260812 01:23:03.076918 25703 tablet_bootstrap.cc:654] T 4424797293c14909b54a4c21c02ff5ec P 6532f4e6bbe74b9eb0a6988d2ca9f943: Neither blocks nor log segments found. Creating new log.
I20260812 01:23:03.077231 25703 log.cc:824] T 4424797293c14909b54a4c21c02ff5ec P 6532f4e6bbe74b9eb0a6988d2ca9f943: Log is configured to *not* fsync() on all Append() calls
I20260812 01:23:03.077534 25702 tablet_bootstrap.cc:492] T 4424797293c14909b54a4c21c02ff5ec P 446790d853a34e6db0de8d5ffa39e90f: No bootstrap required, opened a new log
I20260812 01:23:03.077577 25704 tablet_bootstrap.cc:492] T 4424797293c14909b54a4c21c02ff5ec P 6d117a85b2ff4b4186f3edabd6047f43: Bootstrap starting.
I20260812 01:23:03.077610 25702 ts_tablet_manager.cc:1397] T 4424797293c14909b54a4c21c02ff5ec P 446790d853a34e6db0de8d5ffa39e90f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 01:23:03.077926 25703 tablet_bootstrap.cc:492] T 4424797293c14909b54a4c21c02ff5ec P 6532f4e6bbe74b9eb0a6988d2ca9f943: No bootstrap required, opened a new log
I20260812 01:23:03.078011 25703 ts_tablet_manager.cc:1397] T 4424797293c14909b54a4c21c02ff5ec P 6532f4e6bbe74b9eb0a6988d2ca9f943: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260812 01:23:03.078233 25704 tablet_bootstrap.cc:654] T 4424797293c14909b54a4c21c02ff5ec P 6d117a85b2ff4b4186f3edabd6047f43: Neither blocks nor log segments found. Creating new log.
I20260812 01:23:03.078579 25704 log.cc:824] T 4424797293c14909b54a4c21c02ff5ec P 6d117a85b2ff4b4186f3edabd6047f43: Log is configured to *not* fsync() on all Append() calls
I20260812 01:23:03.079439 25704 tablet_bootstrap.cc:492] T 4424797293c14909b54a4c21c02ff5ec P 6d117a85b2ff4b4186f3edabd6047f43: No bootstrap required, opened a new log
I20260812 01:23:03.079512 25704 ts_tablet_manager.cc:1397] T 4424797293c14909b54a4c21c02ff5ec P 6d117a85b2ff4b4186f3edabd6047f43: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 01:23:03.080401 25703 raft_consensus.cc:348] T 4424797293c14909b54a4c21c02ff5ec P 6532f4e6bbe74b9eb0a6988d2ca9f943 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6532f4e6bbe74b9eb0a6988d2ca9f943" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 34411 } } peers { permanent_uuid: "446790d853a34e6db0de8d5ffa39e90f" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 44621 } } peers { permanent_uuid: "6d117a85b2ff4b4186f3edabd6047f43" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 41843 } }
I20260812 01:23:03.080608 25703 raft_consensus.cc:374] T 4424797293c14909b54a4c21c02ff5ec P 6532f4e6bbe74b9eb0a6988d2ca9f943 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:23:03.080657 25703 raft_consensus.cc:729] T 4424797293c14909b54a4c21c02ff5ec P 6532f4e6bbe74b9eb0a6988d2ca9f943 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6532f4e6bbe74b9eb0a6988d2ca9f943, State: Initialized, Role: FOLLOWER
I20260812 01:23:03.080641 25702 raft_consensus.cc:348] T 4424797293c14909b54a4c21c02ff5ec P 446790d853a34e6db0de8d5ffa39e90f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6532f4e6bbe74b9eb0a6988d2ca9f943" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 34411 } } peers { permanent_uuid: "446790d853a34e6db0de8d5ffa39e90f" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 44621 } } peers { permanent_uuid: "6d117a85b2ff4b4186f3edabd6047f43" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 41843 } }
I20260812 01:23:03.080808 25702 raft_consensus.cc:374] T 4424797293c14909b54a4c21c02ff5ec P 446790d853a34e6db0de8d5ffa39e90f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:23:03.080778 25703 consensus_queue.cc:260] T 4424797293c14909b54a4c21c02ff5ec P 6532f4e6bbe74b9eb0a6988d2ca9f943 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6532f4e6bbe74b9eb0a6988d2ca9f943" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 34411 } } peers { permanent_uuid: "446790d853a34e6db0de8d5ffa39e90f" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 44621 } } peers { permanent_uuid: "6d117a85b2ff4b4186f3edabd6047f43" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 41843 } }
I20260812 01:23:03.080853 25702 raft_consensus.cc:729] T 4424797293c14909b54a4c21c02ff5ec P 446790d853a34e6db0de8d5ffa39e90f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 446790d853a34e6db0de8d5ffa39e90f, State: Initialized, Role: FOLLOWER
I20260812 01:23:03.080989 25702 consensus_queue.cc:260] T 4424797293c14909b54a4c21c02ff5ec P 446790d853a34e6db0de8d5ffa39e90f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6532f4e6bbe74b9eb0a6988d2ca9f943" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 34411 } } peers { permanent_uuid: "446790d853a34e6db0de8d5ffa39e90f" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 44621 } } peers { permanent_uuid: "6d117a85b2ff4b4186f3edabd6047f43" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 41843 } }
I20260812 01:23:03.081037 25703 ts_tablet_manager.cc:1428] T 4424797293c14909b54a4c21c02ff5ec P 6532f4e6bbe74b9eb0a6988d2ca9f943: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260812 01:23:03.081179 25274 heartbeater.cc:503] Master 127.24.0.190:41079 was elected leader, sending a full tablet report...
I20260812 01:23:03.081265 25702 ts_tablet_manager.cc:1428] T 4424797293c14909b54a4c21c02ff5ec P 446790d853a34e6db0de8d5ffa39e90f: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260812 01:23:03.081312 25680 heartbeater.cc:503] Master 127.24.0.190:41079 was elected leader, sending a full tablet report...
I20260812 01:23:03.082188 25704 raft_consensus.cc:348] T 4424797293c14909b54a4c21c02ff5ec P 6d117a85b2ff4b4186f3edabd6047f43 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6532f4e6bbe74b9eb0a6988d2ca9f943" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 34411 } } peers { permanent_uuid: "446790d853a34e6db0de8d5ffa39e90f" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 44621 } } peers { permanent_uuid: "6d117a85b2ff4b4186f3edabd6047f43" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 41843 } }
I20260812 01:23:03.082350 25704 raft_consensus.cc:374] T 4424797293c14909b54a4c21c02ff5ec P 6d117a85b2ff4b4186f3edabd6047f43 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:23:03.082404 25704 raft_consensus.cc:729] T 4424797293c14909b54a4c21c02ff5ec P 6d117a85b2ff4b4186f3edabd6047f43 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d117a85b2ff4b4186f3edabd6047f43, State: Initialized, Role: FOLLOWER
I20260812 01:23:03.082526 25704 consensus_queue.cc:260] T 4424797293c14909b54a4c21c02ff5ec P 6d117a85b2ff4b4186f3edabd6047f43 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6532f4e6bbe74b9eb0a6988d2ca9f943" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 34411 } } peers { permanent_uuid: "446790d853a34e6db0de8d5ffa39e90f" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 44621 } } peers { permanent_uuid: "6d117a85b2ff4b4186f3edabd6047f43" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 41843 } }
I20260812 01:23:03.082796 25704 ts_tablet_manager.cc:1428] T 4424797293c14909b54a4c21c02ff5ec P 6d117a85b2ff4b4186f3edabd6047f43: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260812 01:23:03.082841 25485 heartbeater.cc:503] Master 127.24.0.190:41079 was elected leader, sending a full tablet report...
I20260812 01:23:03.138193 25712 raft_consensus.cc:482] T 4424797293c14909b54a4c21c02ff5ec P 6532f4e6bbe74b9eb0a6988d2ca9f943 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 01:23:03.138376 25712 raft_consensus.cc:504] T 4424797293c14909b54a4c21c02ff5ec P 6532f4e6bbe74b9eb0a6988d2ca9f943 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6532f4e6bbe74b9eb0a6988d2ca9f943" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 34411 } } peers { permanent_uuid: "446790d853a34e6db0de8d5ffa39e90f" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 44621 } } peers { permanent_uuid: "6d117a85b2ff4b4186f3edabd6047f43" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 41843 } }
I20260812 01:23:03.138790 25712 leader_election.cc:288] T 4424797293c14909b54a4c21c02ff5ec P 6532f4e6bbe74b9eb0a6988d2ca9f943 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 446790d853a34e6db0de8d5ffa39e90f (127.24.0.131:44621), 6d117a85b2ff4b4186f3edabd6047f43 (127.24.0.130:41843)
I20260812 01:23:03.142167 25699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:03.139094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.129:60951 (local address 127.24.0.130:41843)
0812 01:23:03.139159 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:03.139160 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:03.139169 (+ 9us) server_negotiation.cc:407] Connection header received
0812 01:23:03.139205 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:03.139206 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:03.139219 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:03.139305 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:23:03.139527 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:03.140297 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:03.141407 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:03.141785 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:03.141843 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:03.141850 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:03.142036 (+ 186us) server_negotiation.cc:299] Negotiation successful
0812 01:23:03.142083 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 01:23:03.142190 25700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:03.139094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.0.129:56129 (local address 127.24.0.131:44621)
0812 01:23:03.139144 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:03.139147 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:03.139365 (+ 218us) server_negotiation.cc:407] Connection header received
0812 01:23:03.139436 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:03.139437 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:03.139449 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:03.139508 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:23:03.139705 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:03.140445 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:03.141430 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:03.141802 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:03.141895 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:03.141903 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:03.142078 (+ 175us) server_negotiation.cc:299] Negotiation successful
0812 01:23:03.142130 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 01:23:03.142179 25279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:03.139003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.130:41843 (local address 127.24.0.129:60951)
0812 01:23:03.139079 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0812 01:23:03.139091 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 01:23:03.139141 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:23:03.139317 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:23:03.139317 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:23:03.139319 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:23:03.139418 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:23:03.139420 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:23:03.140392 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:23:03.140394 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:23:03.141306 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:23:03.141309 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:23:03.141879 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:23:03.141880 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:23:03.141952 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:03.141956 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 01:23:03.142007 (+ 51us) client_negotiation.cc:240] Negotiation successful
0812 01:23:03.142059 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56,"mutex_wait_us":31}
I20260812 01:23:03.142191 25717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:03.139003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.131:44621 (local address 127.24.0.129:56129)
0812 01:23:03.139271 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0812 01:23:03.139282 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 01:23:03.139356 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:23:03.139528 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:23:03.139529 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:23:03.139531 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:23:03.139613 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:23:03.139615 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:23:03.140521 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:23:03.140522 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:23:03.141317 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:23:03.141319 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:23:03.141924 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:23:03.141925 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:23:03.141980 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:03.141983 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 01:23:03.142060 (+ 77us) client_negotiation.cc:240] Negotiation successful
0812 01:23:03.142096 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"mutex_wait_us":43,"thread_start_us":157,"threads_started":1}
I20260812 01:23:03.142560 25619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4424797293c14909b54a4c21c02ff5ec" candidate_uuid: "6532f4e6bbe74b9eb0a6988d2ca9f943" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "446790d853a34e6db0de8d5ffa39e90f" is_pre_election: true
I20260812 01:23:03.142606 25417 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4424797293c14909b54a4c21c02ff5ec" candidate_uuid: "6532f4e6bbe74b9eb0a6988d2ca9f943" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d117a85b2ff4b4186f3edabd6047f43" is_pre_election: true
I20260812 01:23:03.142748 25417 raft_consensus.cc:2454] T 4424797293c14909b54a4c21c02ff5ec P 6d117a85b2ff4b4186f3edabd6047f43 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6532f4e6bbe74b9eb0a6988d2ca9f943 in term 0.
I20260812 01:23:03.142748 25619 raft_consensus.cc:2454] T 4424797293c14909b54a4c21c02ff5ec P 446790d853a34e6db0de8d5ffa39e90f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6532f4e6bbe74b9eb0a6988d2ca9f943 in term 0.
I20260812 01:23:03.142970 25108 leader_election.cc:302] T 4424797293c14909b54a4c21c02ff5ec P 6532f4e6bbe74b9eb0a6988d2ca9f943 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 446790d853a34e6db0de8d5ffa39e90f, 6532f4e6bbe74b9eb0a6988d2ca9f943; no voters:
I20260812 01:23:03.143144 25712 raft_consensus.cc:2781] T 4424797293c14909b54a4c21c02ff5ec P 6532f4e6bbe74b9eb0a6988d2ca9f943 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 01:23:03.143193 25712 raft_consensus.cc:482] T 4424797293c14909b54a4c21c02ff5ec P 6532f4e6bbe74b9eb0a6988d2ca9f943 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 01:23:03.143209 25712 raft_consensus.cc:3037] T 4424797293c14909b54a4c21c02ff5ec P 6532f4e6bbe74b9eb0a6988d2ca9f943 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:23:03.143900 25712 raft_consensus.cc:504] T 4424797293c14909b54a4c21c02ff5ec P 6532f4e6bbe74b9eb0a6988d2ca9f943 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6532f4e6bbe74b9eb0a6988d2ca9f943" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 34411 } } peers { permanent_uuid: "446790d853a34e6db0de8d5ffa39e90f" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 44621 } } peers { permanent_uuid: "6d117a85b2ff4b4186f3edabd6047f43" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 41843 } }
I20260812 01:23:03.144050 25712 leader_election.cc:288] T 4424797293c14909b54a4c21c02ff5ec P 6532f4e6bbe74b9eb0a6988d2ca9f943 [CANDIDATE]: Term 1 election: Requested vote from peers 446790d853a34e6db0de8d5ffa39e90f (127.24.0.131:44621), 6d117a85b2ff4b4186f3edabd6047f43 (127.24.0.130:41843)
I20260812 01:23:03.144217 25619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4424797293c14909b54a4c21c02ff5ec" candidate_uuid: "6532f4e6bbe74b9eb0a6988d2ca9f943" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "446790d853a34e6db0de8d5ffa39e90f"
I20260812 01:23:03.144258 25417 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4424797293c14909b54a4c21c02ff5ec" candidate_uuid: "6532f4e6bbe74b9eb0a6988d2ca9f943" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d117a85b2ff4b4186f3edabd6047f43"
I20260812 01:23:03.144305 25619 raft_consensus.cc:3037] T 4424797293c14909b54a4c21c02ff5ec P 446790d853a34e6db0de8d5ffa39e90f [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:23:03.144321 25417 raft_consensus.cc:3037] T 4424797293c14909b54a4c21c02ff5ec P 6d117a85b2ff4b4186f3edabd6047f43 [term 0 FOLLOWER]: Advancing to term 1
W20260812 01:23:03.145017 25486 tablet.cc:2369] T 4424797293c14909b54a4c21c02ff5ec P 6d117a85b2ff4b4186f3edabd6047f43: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 01:23:03.145257 25619 raft_consensus.cc:2454] T 4424797293c14909b54a4c21c02ff5ec P 446790d853a34e6db0de8d5ffa39e90f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6532f4e6bbe74b9eb0a6988d2ca9f943 in term 1.
I20260812 01:23:03.145257 25417 raft_consensus.cc:2454] T 4424797293c14909b54a4c21c02ff5ec P 6d117a85b2ff4b4186f3edabd6047f43 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6532f4e6bbe74b9eb0a6988d2ca9f943 in term 1.
I20260812 01:23:03.145423 25108 leader_election.cc:302] T 4424797293c14909b54a4c21c02ff5ec P 6532f4e6bbe74b9eb0a6988d2ca9f943 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 446790d853a34e6db0de8d5ffa39e90f, 6532f4e6bbe74b9eb0a6988d2ca9f943; no voters:
I20260812 01:23:03.145526 25712 raft_consensus.cc:2781] T 4424797293c14909b54a4c21c02ff5ec P 6532f4e6bbe74b9eb0a6988d2ca9f943 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:23:03.145694 25712 raft_consensus.cc:686] T 4424797293c14909b54a4c21c02ff5ec P 6532f4e6bbe74b9eb0a6988d2ca9f943 [term 1 LEADER]: Becoming Leader. State: Replica: 6532f4e6bbe74b9eb0a6988d2ca9f943, State: Running, Role: LEADER
I20260812 01:23:03.145798 25712 consensus_queue.cc:237] T 4424797293c14909b54a4c21c02ff5ec P 6532f4e6bbe74b9eb0a6988d2ca9f943 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6532f4e6bbe74b9eb0a6988d2ca9f943" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 34411 } } peers { permanent_uuid: "446790d853a34e6db0de8d5ffa39e90f" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 44621 } } peers { permanent_uuid: "6d117a85b2ff4b4186f3edabd6047f43" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 41843 } }
I20260812 01:23:03.146756 24995 catalog_manager.cc:5495] T 4424797293c14909b54a4c21c02ff5ec P 6532f4e6bbe74b9eb0a6988d2ca9f943 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6532f4e6bbe74b9eb0a6988d2ca9f943 (127.24.0.129). New cstate: current_term: 1 leader_uuid: "6532f4e6bbe74b9eb0a6988d2ca9f943" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6532f4e6bbe74b9eb0a6988d2ca9f943" member_type: VOTER last_known_addr { host: "127.24.0.129" port: 34411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "446790d853a34e6db0de8d5ffa39e90f" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 44621 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d117a85b2ff4b4186f3edabd6047f43" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 41843 } health_report { overall_health: UNKNOWN } } }
I20260812 01:23:03.163486 25694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:03.161049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32802 (local address 127.24.0.129:34411)
0812 01:23:03.161134 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:03.161136 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:03.161144 (+ 8us) server_negotiation.cc:407] Connection header received
0812 01:23:03.161186 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:03.161187 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:03.161209 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:03.161277 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:23:03.161397 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:03.161868 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:03.162629 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:03.162854 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:03.162924 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:03.163085 (+ 161us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:23:03.163203 (+ 118us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:23:03.163276 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:03.163318 (+ 42us) server_negotiation.cc:299] Negotiation successful
0812 01:23:03.163339 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 01:23:03.165609 25417 raft_consensus.cc:1261] T 4424797293c14909b54a4c21c02ff5ec P 6d117a85b2ff4b4186f3edabd6047f43 [term 1 FOLLOWER]: Refusing update from remote peer 6532f4e6bbe74b9eb0a6988d2ca9f943: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:23:03.165621 25619 raft_consensus.cc:1261] T 4424797293c14909b54a4c21c02ff5ec P 446790d853a34e6db0de8d5ffa39e90f [term 1 FOLLOWER]: Refusing update from remote peer 6532f4e6bbe74b9eb0a6988d2ca9f943: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:23:03.165988 25719 consensus_queue.cc:1035] T 4424797293c14909b54a4c21c02ff5ec P 6532f4e6bbe74b9eb0a6988d2ca9f943 [LEADER]: Connected to new peer: Peer: permanent_uuid: "446790d853a34e6db0de8d5ffa39e90f" member_type: VOTER last_known_addr { host: "127.24.0.131" port: 44621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:23:03.166173 25712 consensus_queue.cc:1035] T 4424797293c14909b54a4c21c02ff5ec P 6532f4e6bbe74b9eb0a6988d2ca9f943 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d117a85b2ff4b4186f3edabd6047f43" member_type: VOTER last_known_addr { host: "127.24.0.130" port: 41843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:23:03.169396 25730 mvcc.cc:205] Tried to move back new op lower bound from 7317494919843287040 to 7317494919765901312. Current Snapshot: MvccSnapshot[applied={T|T < 7317494919843287040}]
I20260812 01:23:03.171849 24993 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45952:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:23:03.171967 24993 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45952:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:23:03.172901 24993 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8eb1c87ac6074213b950e85bd694b59f: Sending DeleteTablet for 3 replicas of tablet 4424797293c14909b54a4c21c02ff5ec
I20260812 01:23:03.173378 25389 tablet_service.cc:1514] Processing DeleteTablet for tablet 4424797293c14909b54a4c21c02ff5ec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:23:03 UTC) from {username='slave'} at 127.0.0.1:57804
I20260812 01:23:03.173388 25590 tablet_service.cc:1514] Processing DeleteTablet for tablet 4424797293c14909b54a4c21c02ff5ec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:23:03 UTC) from {username='slave'} at 127.0.0.1:52240
I20260812 01:23:03.173638 25178 tablet_service.cc:1514] Processing DeleteTablet for tablet 4424797293c14909b54a4c21c02ff5ec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:23:03 UTC) from {username='slave'} at 127.0.0.1:32786
I20260812 01:23:03.173662 25742 tablet_replica.cc:330] T 4424797293c14909b54a4c21c02ff5ec P 6d117a85b2ff4b4186f3edabd6047f43: stopping tablet replica
I20260812 01:23:03.173771 25742 raft_consensus.cc:2229] T 4424797293c14909b54a4c21c02ff5ec P 6d117a85b2ff4b4186f3edabd6047f43 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 01:23:03.173880 25742 raft_consensus.cc:2258] T 4424797293c14909b54a4c21c02ff5ec P 6d117a85b2ff4b4186f3edabd6047f43 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:23:03.174049 25740 tablet_replica.cc:330] T 4424797293c14909b54a4c21c02ff5ec P 446790d853a34e6db0de8d5ffa39e90f: stopping tablet replica
I20260812 01:23:03.174154 25740 raft_consensus.cc:2229] T 4424797293c14909b54a4c21c02ff5ec P 446790d853a34e6db0de8d5ffa39e90f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 01:23:03.174263 25740 raft_consensus.cc:2258] T 4424797293c14909b54a4c21c02ff5ec P 446790d853a34e6db0de8d5ffa39e90f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:23:03.174374 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 25083
I20260812 01:23:03.175204 25740 ts_tablet_manager.cc:1905] T 4424797293c14909b54a4c21c02ff5ec P 446790d853a34e6db0de8d5ffa39e90f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:23:03.176899 25740 ts_tablet_manager.cc:1918] T 4424797293c14909b54a4c21c02ff5ec P 446790d853a34e6db0de8d5ffa39e90f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 01:23:03.176977 25740 log.cc:1196] T 4424797293c14909b54a4c21c02ff5ec P 446790d853a34e6db0de8d5ffa39e90f: Deleting WAL directory at /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786497534195564-24578-0/minicluster-data/ts-2/wal/wals/4424797293c14909b54a4c21c02ff5ec
I20260812 01:23:03.177199 25740 ts_tablet_manager.cc:1939] T 4424797293c14909b54a4c21c02ff5ec P 446790d853a34e6db0de8d5ffa39e90f: Deleting consensus metadata
I20260812 01:23:03.177618 24973 catalog_manager.cc:4841] TS 446790d853a34e6db0de8d5ffa39e90f (127.24.0.131:44621): tablet 4424797293c14909b54a4c21c02ff5ec (table test-table [id=e66f6f2c90654d2c8f48e8ab098ae829]) successfully deleted
I20260812 01:23:03.189438 25742 ts_tablet_manager.cc:1905] T 4424797293c14909b54a4c21c02ff5ec P 6d117a85b2ff4b4186f3edabd6047f43: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:23:03.191144 25742 ts_tablet_manager.cc:1918] T 4424797293c14909b54a4c21c02ff5ec P 6d117a85b2ff4b4186f3edabd6047f43: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 01:23:03.191247 25742 log.cc:1196] T 4424797293c14909b54a4c21c02ff5ec P 6d117a85b2ff4b4186f3edabd6047f43: Deleting WAL directory at /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786497534195564-24578-0/minicluster-data/ts-1/wal/wals/4424797293c14909b54a4c21c02ff5ec
I20260812 01:23:03.191517 25742 ts_tablet_manager.cc:1939] T 4424797293c14909b54a4c21c02ff5ec P 6d117a85b2ff4b4186f3edabd6047f43: Deleting consensus metadata
I20260812 01:23:03.192286 24972 catalog_manager.cc:4841] TS 6d117a85b2ff4b4186f3edabd6047f43 (127.24.0.130:41843): tablet 4424797293c14909b54a4c21c02ff5ec (table test-table [id=e66f6f2c90654d2c8f48e8ab098ae829]) successfully deleted
W20260812 01:23:03.194231 24973 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.0.129:34411 (error 108)
I20260812 01:23:03.194783 25695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:03.194623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.129:34411 (local address 127.0.0.1:32812)
0812 01:23:03.194693 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0812 01:23:03.194751 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.129:34411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260812 01:23:03.194814 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 25281
W20260812 01:23:03.194949 24973 catalog_manager.cc:4584] TS 6532f4e6bbe74b9eb0a6988d2ca9f943 (127.24.0.129:34411): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4424797293c14909b54a4c21c02ff5ec: Network error: Client connection negotiation failed: client connection to 127.24.0.129:34411: connect: Connection refused (error 111)
I20260812 01:23:03.207525 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 25489
I20260812 01:23:03.218254 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 24946
2026-08-12T01:23:03Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (781 ms)
[----------] 6 tests from EncryptionPolicyTest (9815 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786497534195564-24578-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 01:23:03 dist-test-slave-t3q3 krb5kdc[25753](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 01:23:03 dist-test-slave-t3q3 krb5kdc[25753](info): set up 2 sockets
Aug 12 01:23:03 dist-test-slave-t3q3 krb5kdc[25753](info): commencing operation
krb5kdc: starting...
W20260812 01:23:04.518221 24578 mini_kdc.cc:121] Time spent starting KDC: real 1.268s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786497534195564-24578-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786497534195564-24578-0/krb5kdc/test-user.keytab.
Aug 12 01:23:04 dist-test-slave-t3q3 krb5kdc[25753](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497784, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T01:23:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:23:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786497534195564-24578-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786497534195564-24578-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 01:23:04.642758 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786497534195564-24578-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786497534195564-24578-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786497534195564-24578-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37863
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786497534195564-24578-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37863
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:23:04.726722 25769 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:23:04.726984 25769 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:23:04.727036 25769 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:23:04.728482 25769 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:23:04.728551 25769 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:23:04.728578 25769 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:23:04.728601 25769 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:23:04.728632 25769 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:23:04.731453 25769 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786497534195564-24578-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786497534195564-24578-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37863
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786497534195564-24578-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37863
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25769
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786497534195564-24578-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:23:04.731807 25769 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:23:04.732113 25769 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:23:04.737646 25781 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:23:04.737638 25779 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:23:04.737638 25775 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:23:04.737812 25769 server_base.cc:1034] running on GCE node
I20260812 01:23:04.738114 25769 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:23:04.738489 25769 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:23:04.739643 25769 hybrid_clock.cc:648] HybridClock initialized: now 1786497784739622 us; error 40 us; skew 500 ppm
Aug 12 01:23:04 dist-test-slave-t3q3 krb5kdc[25753](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497784, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:23:04.742285 25769 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 01:23:04.742678 25769 webserver.cc:460] Webserver started at http://127.0.0.1:45615/ using document root <none> and password file <none>
I20260812 01:23:04.742880 25769 fs_manager.cc:359] Metadata directory not provided
I20260812 01:23:04.742920 25769 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:23:04.743010 25769 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:23:04.744028 25769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786497534195564-24578-0/minicluster-data/master-0/data/instance:
uuid: "5e6bde7abfa8425e8ce145b2fd72b4bf"
format_stamp: "Formatted at 2026-08-12 01:23:04 on dist-test-slave-t3q3"
server_key: "fc544ccbd24bb98d058a58c387a7285b"
server_key_iv: "22d2f97e5efd837c8b5a4b32502aaa60"
server_key_version: "encryptionkey@0"
I20260812 01:23:04.744437 25769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786497534195564-24578-0/minicluster-data/master-0/wal/instance:
uuid: "5e6bde7abfa8425e8ce145b2fd72b4bf"
format_stamp: "Formatted at 2026-08-12 01:23:04 on dist-test-slave-t3q3"
server_key: "fc544ccbd24bb98d058a58c387a7285b"
server_key_iv: "22d2f97e5efd837c8b5a4b32502aaa60"
server_key_version: "encryptionkey@0"
I20260812 01:23:04.745774 25769 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 01:23:04.746657 25791 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:23:04.746877 25769 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 01:23:04.746974 25769 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786497534195564-24578-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786497534195564-24578-0/minicluster-data/master-0/wal
uuid: "5e6bde7abfa8425e8ce145b2fd72b4bf"
format_stamp: "Formatted at 2026-08-12 01:23:04 on dist-test-slave-t3q3"
server_key: "fc544ccbd24bb98d058a58c387a7285b"
server_key_iv: "22d2f97e5efd837c8b5a4b32502aaa60"
server_key_version: "encryptionkey@0"
I20260812 01:23:04.747043 25769 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786497534195564-24578-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786497534195564-24578-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786497534195564-24578-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:23:04.763969 25769 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:23:04.766179 25769 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:23:04.766305 25769 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:23:04.770764 25769 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37863
I20260812 01:23:04.770818 25875 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37863 every 8 connection(s)
I20260812 01:23:04.771214 25769 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
I20260812 01:23:04.772221 25876 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:23:04.774513 25876 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e6bde7abfa8425e8ce145b2fd72b4bf: Bootstrap starting.
I20260812 01:23:04.775071 25876 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5e6bde7abfa8425e8ce145b2fd72b4bf: Neither blocks nor log segments found. Creating new log.
I20260812 01:23:04.775310 25876 log.cc:824] T 00000000000000000000000000000000 P 5e6bde7abfa8425e8ce145b2fd72b4bf: Log is configured to *not* fsync() on all Append() calls
I20260812 01:23:04.776137 25876 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e6bde7abfa8425e8ce145b2fd72b4bf: No bootstrap required, opened a new log
I20260812 01:23:04.777717 25876 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5e6bde7abfa8425e8ce145b2fd72b4bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e6bde7abfa8425e8ce145b2fd72b4bf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37863 } }
I20260812 01:23:04.777854 25876 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5e6bde7abfa8425e8ce145b2fd72b4bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:23:04.777877 25876 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5e6bde7abfa8425e8ce145b2fd72b4bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e6bde7abfa8425e8ce145b2fd72b4bf, State: Initialized, Role: FOLLOWER
I20260812 01:23:04.778013 25876 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5e6bde7abfa8425e8ce145b2fd72b4bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e6bde7abfa8425e8ce145b2fd72b4bf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37863 } }
I20260812 01:23:04.778097 25876 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5e6bde7abfa8425e8ce145b2fd72b4bf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:23:04.778119 25876 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5e6bde7abfa8425e8ce145b2fd72b4bf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:23:04.778158 25876 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5e6bde7abfa8425e8ce145b2fd72b4bf [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:23:04.778657 25876 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5e6bde7abfa8425e8ce145b2fd72b4bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e6bde7abfa8425e8ce145b2fd72b4bf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37863 } }
I20260812 01:23:04.778731 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 25769
I20260812 01:23:04.778741 25876 leader_election.cc:302] T 00000000000000000000000000000000 P 5e6bde7abfa8425e8ce145b2fd72b4bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5e6bde7abfa8425e8ce145b2fd72b4bf; no voters:
I20260812 01:23:04.778869 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786497534195564-24578-0/minicluster-data/master-0/wal/instance
I20260812 01:23:04.778955 25876 leader_election.cc:288] T 00000000000000000000000000000000 P 5e6bde7abfa8425e8ce145b2fd72b4bf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 01:23:04.778976 25879 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5e6bde7abfa8425e8ce145b2fd72b4bf [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:23:04.779075 24578 external_mini_cluster.cc:1428] Setting key d67e66e1f86193a72fa072e9ad8d0271
I20260812 01:23:04.779100 25879 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5e6bde7abfa8425e8ce145b2fd72b4bf [term 1 LEADER]: Becoming Leader. State: Replica: 5e6bde7abfa8425e8ce145b2fd72b4bf, State: Running, Role: LEADER
I20260812 01:23:04.779177 25879 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5e6bde7abfa8425e8ce145b2fd72b4bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e6bde7abfa8425e8ce145b2fd72b4bf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37863 } }
I20260812 01:23:04.779537 25882 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e6bde7abfa8425e8ce145b2fd72b4bf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5e6bde7abfa8425e8ce145b2fd72b4bf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e6bde7abfa8425e8ce145b2fd72b4bf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37863 } } }
I20260812 01:23:04.779680 25881 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e6bde7abfa8425e8ce145b2fd72b4bf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5e6bde7abfa8425e8ce145b2fd72b4bf. Latest consensus state: current_term: 1 leader_uuid: "5e6bde7abfa8425e8ce145b2fd72b4bf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e6bde7abfa8425e8ce145b2fd72b4bf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37863 } } }
I20260812 01:23:04.779696 25876 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5e6bde7abfa8425e8ce145b2fd72b4bf [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:23:04.779742 25881 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e6bde7abfa8425e8ce145b2fd72b4bf [sys.catalog]: This master's current role is: LEADER
I20260812 01:23:04.779712 25882 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e6bde7abfa8425e8ce145b2fd72b4bf [sys.catalog]: This master's current role is: LEADER
I20260812 01:23:04.780268 25889 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:23:04.780856 25889 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:23:04.782668 25889 catalog_manager.cc:1347] Generated new cluster ID: 9fae6cf266d746bbae9f450258bb7ae4
I20260812 01:23:04.782747 25889 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 01:23:04 dist-test-slave-t3q3 krb5kdc[25753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497784, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 01:23:04.791549 25887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:04.779684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55281 (local address 127.0.0.1:37863)
0812 01:23:04.779869 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:04.779881 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:04.780131 (+ 250us) server_negotiation.cc:407] Connection header received
0812 01:23:04.780265 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:04.780273 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:04.780330 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:04.781504 (+ 1174us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:23:04.781614 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:04.782996 (+ 1382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:04.784039 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:04.784719 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:04.784838 (+ 119us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:04.787534 (+ 2696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:23:04.787539 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:23:04.787539 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:23:04.787552 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:23:04.789728 (+ 2176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:23:04.790168 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:23:04.790171 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:23:04.790172 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:23:04.790307 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:23:04.790551 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:23:04.790552 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:23:04.790552 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:23:04.790899 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:23:04.791098 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:04.791181 (+ 83us) server_negotiation.cc:299] Negotiation successful
0812 01:23:04.791296 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":71,"threads_started":1}
I20260812 01:23:04.792591 24578 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 01:23:04.793498 25889 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:23:04.794215 25889 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:23:04.798218 25889 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5e6bde7abfa8425e8ce145b2fd72b4bf: Generated new TSK 0
I20260812 01:23:04.798413 25889 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 01:23:04.801632 25887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:04.794103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44799 (local address 127.0.0.1:37863)
0812 01:23:04.794169 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:04.794171 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:04.794439 (+ 268us) server_negotiation.cc:407] Connection header received
0812 01:23:04.794486 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:04.794486 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:04.794496 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:04.794536 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:23:04.795201 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:04.795954 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:04.797035 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:04.797387 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:04.797449 (+ 62us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:04.798912 (+ 1463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:23:04.798913 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:23:04.798913 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:23:04.798919 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:23:04.800461 (+ 1542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:23:04.800883 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:23:04.800883 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:23:04.800884 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:23:04.800935 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:23:04.801190 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:23:04.801190 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:23:04.801190 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:23:04.801285 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:23:04.801365 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:04.801526 (+ 161us) server_negotiation.cc:299] Negotiation successful
0812 01:23:04.801575 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 01:23:04.804046 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 25769
W20260812 01:23:04.817283 25751 connection.cc:535] client connection to 127.0.0.1:37863 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37863): Connection reset by peer (error 104)
2026-08-12T01:23:04Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1581 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786497534195564-24578-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 01:23:04 dist-test-slave-t3q3 krb5kdc[25922](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 01:23:04 dist-test-slave-t3q3 krb5kdc[25922](info): set up 2 sockets
Aug 12 01:23:04 dist-test-slave-t3q3 krb5kdc[25922](info): commencing operation
krb5kdc: starting...
W20260812 01:23:06.109030 24578 mini_kdc.cc:121] Time spent starting KDC: real 1.278s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786497534195564-24578-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786497534195564-24578-0/krb5kdc/test-user.keytab.
Aug 12 01:23:06 dist-test-slave-t3q3 krb5kdc[25922](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497786, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T01:23:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:23:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786497534195564-24578-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786497534195564-24578-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 01:23:06.236517 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786497534195564-24578-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786497534195564-24578-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786497534195564-24578-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43273
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786497534195564-24578-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43273
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:23:06.318221 25943 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:23:06.318429 25943 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:23:06.318452 25943 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:23:06.319638 25943 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:23:06.319684 25943 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:23:06.319697 25943 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:23:06.319710 25943 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:23:06.319722 25943 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:23:06.321254 25943 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786497534195564-24578-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786497534195564-24578-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43273
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786497534195564-24578-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43273
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25943
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786497534195564-24578-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:23:06.321511 25943 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:23:06.321753 25943 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:23:06.326610 25950 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:23:06.326617 25949 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:23:06.326656 25943 server_base.cc:1034] running on GCE node
W20260812 01:23:06.326617 25955 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:23:06.327096 25943 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:23:06.327394 25943 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:23:06.328536 25943 hybrid_clock.cc:648] HybridClock initialized: now 1786497786328524 us; error 33 us; skew 500 ppm
Aug 12 01:23:06 dist-test-slave-t3q3 krb5kdc[25922](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497786, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:23:06.331059 25943 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 01:23:06.331516 25943 webserver.cc:460] Webserver started at http://127.0.0.1:40113/ using document root <none> and password file <none>
I20260812 01:23:06.331754 25943 fs_manager.cc:359] Metadata directory not provided
I20260812 01:23:06.331799 25943 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:23:06.331964 25943 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:23:06.333031 25943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786497534195564-24578-0/minicluster-data/master-0/data/instance:
uuid: "936578b7be2441f3a9b2e009565ced48"
format_stamp: "Formatted at 2026-08-12 01:23:06 on dist-test-slave-t3q3"
server_key: "f6f87beb442cc4c03c08b50e52ec592a"
server_key_iv: "2e2b27d1f38ce09fea56abc5f0193689"
server_key_version: "encryptionkey@0"
I20260812 01:23:06.333398 25943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786497534195564-24578-0/minicluster-data/master-0/wal/instance:
uuid: "936578b7be2441f3a9b2e009565ced48"
format_stamp: "Formatted at 2026-08-12 01:23:06 on dist-test-slave-t3q3"
server_key: "f6f87beb442cc4c03c08b50e52ec592a"
server_key_iv: "2e2b27d1f38ce09fea56abc5f0193689"
server_key_version: "encryptionkey@0"
I20260812 01:23:06.334731 25943 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 01:23:06.335490 25962 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:23:06.335688 25943 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:23:06.335799 25943 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786497534195564-24578-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786497534195564-24578-0/minicluster-data/master-0/wal
uuid: "936578b7be2441f3a9b2e009565ced48"
format_stamp: "Formatted at 2026-08-12 01:23:06 on dist-test-slave-t3q3"
server_key: "f6f87beb442cc4c03c08b50e52ec592a"
server_key_iv: "2e2b27d1f38ce09fea56abc5f0193689"
server_key_version: "encryptionkey@0"
I20260812 01:23:06.335873 25943 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786497534195564-24578-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786497534195564-24578-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786497534195564-24578-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:23:06.356593 25943 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:23:06.358464 25943 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:23:06.358646 25943 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:23:06.362936 25943 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43273
I20260812 01:23:06.362977 26041 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43273 every 8 connection(s)
I20260812 01:23:06.363370 25943 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
I20260812 01:23:06.364282 26046 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:23:06.366549 26046 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 936578b7be2441f3a9b2e009565ced48: Bootstrap starting.
I20260812 01:23:06.367100 26046 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 936578b7be2441f3a9b2e009565ced48: Neither blocks nor log segments found. Creating new log.
I20260812 01:23:06.367341 26046 log.cc:824] T 00000000000000000000000000000000 P 936578b7be2441f3a9b2e009565ced48: Log is configured to *not* fsync() on all Append() calls
I20260812 01:23:06.368018 26046 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 936578b7be2441f3a9b2e009565ced48: No bootstrap required, opened a new log
I20260812 01:23:06.369645 26046 raft_consensus.cc:348] T 00000000000000000000000000000000 P 936578b7be2441f3a9b2e009565ced48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "936578b7be2441f3a9b2e009565ced48" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43273 } }
I20260812 01:23:06.369802 26046 raft_consensus.cc:374] T 00000000000000000000000000000000 P 936578b7be2441f3a9b2e009565ced48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:23:06.369825 26046 raft_consensus.cc:729] T 00000000000000000000000000000000 P 936578b7be2441f3a9b2e009565ced48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 936578b7be2441f3a9b2e009565ced48, State: Initialized, Role: FOLLOWER
I20260812 01:23:06.369925 26046 consensus_queue.cc:260] T 00000000000000000000000000000000 P 936578b7be2441f3a9b2e009565ced48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "936578b7be2441f3a9b2e009565ced48" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43273 } }
I20260812 01:23:06.370007 26046 raft_consensus.cc:388] T 00000000000000000000000000000000 P 936578b7be2441f3a9b2e009565ced48 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:23:06.370043 26046 raft_consensus.cc:482] T 00000000000000000000000000000000 P 936578b7be2441f3a9b2e009565ced48 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:23:06.370095 26046 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 936578b7be2441f3a9b2e009565ced48 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:23:06.370630 26046 raft_consensus.cc:504] T 00000000000000000000000000000000 P 936578b7be2441f3a9b2e009565ced48 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "936578b7be2441f3a9b2e009565ced48" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43273 } }
I20260812 01:23:06.370767 26046 leader_election.cc:302] T 00000000000000000000000000000000 P 936578b7be2441f3a9b2e009565ced48 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 936578b7be2441f3a9b2e009565ced48; no voters:
I20260812 01:23:06.370955 26046 leader_election.cc:288] T 00000000000000000000000000000000 P 936578b7be2441f3a9b2e009565ced48 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 01:23:06.370999 26052 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 936578b7be2441f3a9b2e009565ced48 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:23:06.371196 26052 raft_consensus.cc:686] T 00000000000000000000000000000000 P 936578b7be2441f3a9b2e009565ced48 [term 1 LEADER]: Becoming Leader. State: Replica: 936578b7be2441f3a9b2e009565ced48, State: Running, Role: LEADER
I20260812 01:23:06.371284 26046 sys_catalog.cc:564] T 00000000000000000000000000000000 P 936578b7be2441f3a9b2e009565ced48 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:23:06.371537 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 25943
I20260812 01:23:06.371662 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786497534195564-24578-0/minicluster-data/master-0/wal/instance
I20260812 01:23:06.371868 24578 external_mini_cluster.cc:1428] Setting key dcd251c16e06eeea16229f2478c67300
I20260812 01:23:06.371287 26052 consensus_queue.cc:237] T 00000000000000000000000000000000 P 936578b7be2441f3a9b2e009565ced48 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "936578b7be2441f3a9b2e009565ced48" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43273 } }
I20260812 01:23:06.372697 26053 sys_catalog.cc:455] T 00000000000000000000000000000000 P 936578b7be2441f3a9b2e009565ced48 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "936578b7be2441f3a9b2e009565ced48" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "936578b7be2441f3a9b2e009565ced48" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43273 } } }
I20260812 01:23:06.372823 26053 sys_catalog.cc:458] T 00000000000000000000000000000000 P 936578b7be2441f3a9b2e009565ced48 [sys.catalog]: This master's current role is: LEADER
I20260812 01:23:06.373049 26055 sys_catalog.cc:455] T 00000000000000000000000000000000 P 936578b7be2441f3a9b2e009565ced48 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 936578b7be2441f3a9b2e009565ced48. Latest consensus state: current_term: 1 leader_uuid: "936578b7be2441f3a9b2e009565ced48" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "936578b7be2441f3a9b2e009565ced48" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43273 } } }
I20260812 01:23:06.373111 26055 sys_catalog.cc:458] T 00000000000000000000000000000000 P 936578b7be2441f3a9b2e009565ced48 [sys.catalog]: This master's current role is: LEADER
W20260812 01:23:06.373383 26076 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 936578b7be2441f3a9b2e009565ced48: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 01:23:06.373432 26076 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 01:23:06.373507 26077 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:23:06.373668 26077 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:23:06.374939 26077 catalog_manager.cc:1347] Generated new cluster ID: 207ab97275c74a28b052f26badc25254
I20260812 01:23:06.374996 26077 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 01:23:06.378950 26081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:06.373245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41971 (local address 127.0.0.1:43273)
0812 01:23:06.373502 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:06.373514 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:06.373548 (+ 34us) server_negotiation.cc:407] Connection header received
0812 01:23:06.373680 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:06.373687 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:06.373748 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:06.373912 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:23:06.374147 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:06.375576 (+ 1429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:06.376693 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:06.377318 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:06.377436 (+ 118us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:06.378201 (+ 765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:23:06.378205 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:23:06.378205 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:23:06.378430 (+ 225us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:23:06.378472 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:23:06.378491 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:23:06.378585 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:06.378676 (+ 91us) server_negotiation.cc:299] Negotiation successful
0812 01:23:06.378724 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":203,"thread_start_us":77,"threads_started":1}
I20260812 01:23:06.380121 24578 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 01:23:06.387768 26081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:06.381600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41197 (local address 127.0.0.1:43273)
0812 01:23:06.381641 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:06.381643 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:06.384260 (+ 2617us) server_negotiation.cc:407] Connection header received
0812 01:23:06.384312 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:06.384313 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:06.384324 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:06.384389 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:23:06.384521 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:06.385291 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:06.386276 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:06.386644 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:06.386717 (+ 73us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:06.387492 (+ 775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:23:06.387493 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:23:06.387493 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:23:06.387498 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:23:06.387515 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:23:06.387520 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:23:06.387585 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:06.387663 (+ 78us) server_negotiation.cc:299] Negotiation successful
0812 01:23:06.387677 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260812 01:23:06.393831 26077 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:23:06.394603 26077 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:23:06.405230 26077 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 936578b7be2441f3a9b2e009565ced48: Generated new TSK 0
I20260812 01:23:06.405484 26077 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 01:23:06.415992 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 25943
W20260812 01:23:06.427235 25920 connection.cc:535] client connection to 127.0.0.1:43273 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43273): Connection reset by peer (error 104)
2026-08-12T01:23:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1610 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786497534195564-24578-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 01:23:06 dist-test-slave-t3q3 krb5kdc[26092](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 01:23:06 dist-test-slave-t3q3 krb5kdc[26092](info): set up 2 sockets
Aug 12 01:23:06 dist-test-slave-t3q3 krb5kdc[26092](info): commencing operation
krb5kdc: starting...
W20260812 01:23:07.678447 24578 mini_kdc.cc:121] Time spent starting KDC: real 1.237s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786497534195564-24578-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786497534195564-24578-0/krb5kdc/test-user.keytab.
Aug 12 01:23:07 dist-test-slave-t3q3 krb5kdc[26092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497787, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T01:23:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:23:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786497534195564-24578-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786497534195564-24578-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 01:23:07.804874 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786497534195564-24578-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786497534195564-24578-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786497534195564-24578-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42671
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786497534195564-24578-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42671
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:23:07.889411 26111 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:23:07.889634 26111 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:23:07.889657 26111 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:23:07.890822 26111 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:23:07.890866 26111 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:23:07.890880 26111 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:23:07.890893 26111 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:23:07.890905 26111 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:23:07.892357 26111 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786497534195564-24578-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786497534195564-24578-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42671
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786497534195564-24578-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42671
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26111
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786497534195564-24578-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:23:07.892675 26111 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:23:07.892905 26111 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:23:07.897861 26120 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:23:07.897874 26124 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:23:07.897861 26122 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:23:07.898109 26111 server_base.cc:1034] running on GCE node
I20260812 01:23:07.898437 26111 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:23:07.898705 26111 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:23:07.899847 26111 hybrid_clock.cc:648] HybridClock initialized: now 1786497787899835 us; error 30 us; skew 500 ppm
Aug 12 01:23:07 dist-test-slave-t3q3 krb5kdc[26092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497787, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:23:07.902504 26111 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 01:23:07.903010 26111 webserver.cc:460] Webserver started at http://127.0.0.1:37275/ using document root <none> and password file <none>
I20260812 01:23:07.903244 26111 fs_manager.cc:359] Metadata directory not provided
I20260812 01:23:07.903290 26111 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:23:07.903457 26111 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:23:07.904603 26111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786497534195564-24578-0/minicluster-data/master-0/data/instance:
uuid: "2187effda98040fbb28e72940a509970"
format_stamp: "Formatted at 2026-08-12 01:23:07 on dist-test-slave-t3q3"
server_key: "48554b51267f387468b5bbe72fed5677"
server_key_iv: "c120daca7c40872308b38330535a363f"
server_key_version: "encryptionkey@0"
I20260812 01:23:07.904999 26111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786497534195564-24578-0/minicluster-data/master-0/wal/instance:
uuid: "2187effda98040fbb28e72940a509970"
format_stamp: "Formatted at 2026-08-12 01:23:07 on dist-test-slave-t3q3"
server_key: "48554b51267f387468b5bbe72fed5677"
server_key_iv: "c120daca7c40872308b38330535a363f"
server_key_version: "encryptionkey@0"
I20260812 01:23:07.906342 26111 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 01:23:07.907058 26138 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:23:07.907245 26111 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 01:23:07.907356 26111 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786497534195564-24578-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786497534195564-24578-0/minicluster-data/master-0/wal
uuid: "2187effda98040fbb28e72940a509970"
format_stamp: "Formatted at 2026-08-12 01:23:07 on dist-test-slave-t3q3"
server_key: "48554b51267f387468b5bbe72fed5677"
server_key_iv: "c120daca7c40872308b38330535a363f"
server_key_version: "encryptionkey@0"
I20260812 01:23:07.907434 26111 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786497534195564-24578-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786497534195564-24578-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786497534195564-24578-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:23:07.910949 26111 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:23:07.913141 26111 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:23:07.913300 26111 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:23:07.917514 26111 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42671
I20260812 01:23:07.917546 26232 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42671 every 8 connection(s)
I20260812 01:23:07.917891 26111 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
I20260812 01:23:07.918907 26233 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:23:07.920288 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 26111
I20260812 01:23:07.920419 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786497534195564-24578-0/minicluster-data/master-0/wal/instance
I20260812 01:23:07.920629 24578 external_mini_cluster.cc:1428] Setting key 627f617b0c55125e429f91cd05c77c5d
I20260812 01:23:07.921830 26233 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2187effda98040fbb28e72940a509970: Bootstrap starting.
I20260812 01:23:07.922576 26233 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2187effda98040fbb28e72940a509970: Neither blocks nor log segments found. Creating new log.
I20260812 01:23:07.922959 26233 log.cc:824] T 00000000000000000000000000000000 P 2187effda98040fbb28e72940a509970: Log is configured to *not* fsync() on all Append() calls
I20260812 01:23:07.923389 26238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:07.921183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54699 (local address 127.0.0.1:42671)
0812 01:23:07.921396 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:07.921406 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:07.921444 (+ 38us) server_negotiation.cc:407] Connection header received
0812 01:23:07.922007 (+ 563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:07.922016 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:07.922061 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:07.922234 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:23:07.922880 (+ 646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:23:07.922885 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:23:07.922886 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:23:07.922890 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:23:07.922939 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:23:07.922953 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:23:07.923052 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:07.923127 (+ 75us) server_negotiation.cc:299] Negotiation successful
0812 01:23:07.923172 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":85,"threads_started":1}
I20260812 01:23:07.923795 26233 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2187effda98040fbb28e72940a509970: No bootstrap required, opened a new log
I20260812 01:23:07.924455 24578 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 01:23:07.926139 26233 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2187effda98040fbb28e72940a509970 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2187effda98040fbb28e72940a509970" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42671 } }
I20260812 01:23:07.926303 26233 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2187effda98040fbb28e72940a509970 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:23:07.926352 26233 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2187effda98040fbb28e72940a509970 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2187effda98040fbb28e72940a509970, State: Initialized, Role: FOLLOWER
I20260812 01:23:07.926479 26233 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2187effda98040fbb28e72940a509970 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2187effda98040fbb28e72940a509970" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42671 } }
I20260812 01:23:07.926587 26233 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2187effda98040fbb28e72940a509970 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:23:07.926640 26233 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2187effda98040fbb28e72940a509970 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:23:07.926702 26233 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2187effda98040fbb28e72940a509970 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:23:07.927044 26238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:07.925869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37115 (local address 127.0.0.1:42671)
0812 01:23:07.925903 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:07.925905 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:07.926032 (+ 127us) server_negotiation.cc:407] Connection header received
0812 01:23:07.926105 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:07.926106 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:07.926111 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:07.926187 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:23:07.926805 (+ 618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:23:07.926806 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:23:07.926807 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:23:07.926812 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:23:07.926828 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:23:07.926832 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:23:07.926889 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:07.926962 (+ 73us) server_negotiation.cc:299] Negotiation successful
0812 01:23:07.926976 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260812 01:23:07.927505 26233 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2187effda98040fbb28e72940a509970 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2187effda98040fbb28e72940a509970" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42671 } }
I20260812 01:23:07.927646 26233 leader_election.cc:302] T 00000000000000000000000000000000 P 2187effda98040fbb28e72940a509970 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2187effda98040fbb28e72940a509970; no voters:
I20260812 01:23:07.927824 26233 leader_election.cc:288] T 00000000000000000000000000000000 P 2187effda98040fbb28e72940a509970 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 01:23:07.927845 26246 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2187effda98040fbb28e72940a509970 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:23:07.927991 26246 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2187effda98040fbb28e72940a509970 [term 1 LEADER]: Becoming Leader. State: Replica: 2187effda98040fbb28e72940a509970, State: Running, Role: LEADER
I20260812 01:23:07.928139 26246 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2187effda98040fbb28e72940a509970 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2187effda98040fbb28e72940a509970" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42671 } }
I20260812 01:23:07.928192 26233 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2187effda98040fbb28e72940a509970 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:23:07.928522 26248 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2187effda98040fbb28e72940a509970 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2187effda98040fbb28e72940a509970. Latest consensus state: current_term: 1 leader_uuid: "2187effda98040fbb28e72940a509970" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2187effda98040fbb28e72940a509970" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42671 } } }
I20260812 01:23:07.928520 26247 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2187effda98040fbb28e72940a509970 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2187effda98040fbb28e72940a509970" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2187effda98040fbb28e72940a509970" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42671 } } }
I20260812 01:23:07.928712 26248 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2187effda98040fbb28e72940a509970 [sys.catalog]: This master's current role is: LEADER
I20260812 01:23:07.928769 26247 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2187effda98040fbb28e72940a509970 [sys.catalog]: This master's current role is: LEADER
I20260812 01:23:07.929293 26261 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:23:07.929817 26261 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:23:07.931453 26261 catalog_manager.cc:1347] Generated new cluster ID: 2ae21eeacee3484789d91082f3c01417
I20260812 01:23:07.931516 26261 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 01:23:07.939327 26261 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:23:07.939513 26261 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:23:07.942466 26261 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2187effda98040fbb28e72940a509970: Generated new TSK 0
I20260812 01:23:07.942688 26261 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 01:23:07.945123 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 26111
2026-08-12T01:23:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1526 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786497534195564-24578-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 01:23:08 dist-test-slave-t3q3 krb5kdc[26278](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 01:23:08 dist-test-slave-t3q3 krb5kdc[26278](info): set up 2 sockets
Aug 12 01:23:08 dist-test-slave-t3q3 krb5kdc[26278](info): commencing operation
krb5kdc: starting...
W20260812 01:23:09.208120 24578 mini_kdc.cc:121] Time spent starting KDC: real 1.239s user 0.005s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786497534195564-24578-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786497534195564-24578-0/krb5kdc/test-user.keytab.
Aug 12 01:23:09 dist-test-slave-t3q3 krb5kdc[26278](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497789, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T01:23:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:23:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786497534195564-24578-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786497534195564-24578-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 01:23:09.332904 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786497534195564-24578-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786497534195564-24578-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786497534195564-24578-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45793
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786497534195564-24578-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45793
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:23:09.417568 26297 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:23:09.417800 26297 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:23:09.417824 26297 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:23:09.418973 26297 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:23:09.419018 26297 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:23:09.419032 26297 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:23:09.419051 26297 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:23:09.419063 26297 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:23:09.420514 26297 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786497534195564-24578-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786497534195564-24578-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45793
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786497534195564-24578-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45793
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26297
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786497534195564-24578-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:23:09.420706 26297 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:23:09.420881 26297 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:23:09.425694 26305 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:23:09.425717 26307 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:23:09.425889 26297 server_base.cc:1034] running on GCE node
W20260812 01:23:09.425683 26304 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:23:09.426227 26297 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:23:09.426525 26297 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:23:09.427682 26297 hybrid_clock.cc:648] HybridClock initialized: now 1786497789427672 us; error 28 us; skew 500 ppm
Aug 12 01:23:09 dist-test-slave-t3q3 krb5kdc[26278](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497789, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:23:09.435082 26297 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 01:23:09.435563 26297 webserver.cc:460] Webserver started at http://127.0.0.1:46863/ using document root <none> and password file <none>
I20260812 01:23:09.435823 26297 fs_manager.cc:359] Metadata directory not provided
I20260812 01:23:09.435870 26297 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:23:09.436043 26297 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:23:09.437179 26297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786497534195564-24578-0/minicluster-data/master-0/data/instance:
uuid: "e550a01aaebf424a820d877f587c8ce5"
format_stamp: "Formatted at 2026-08-12 01:23:09 on dist-test-slave-t3q3"
server_key: "63f55870b2670daf8e66de46877c504e"
server_key_iv: "345ce9998386c3fb8c41004fd128ac4c"
server_key_version: "encryptionkey@0"
I20260812 01:23:09.437547 26297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786497534195564-24578-0/minicluster-data/master-0/wal/instance:
uuid: "e550a01aaebf424a820d877f587c8ce5"
format_stamp: "Formatted at 2026-08-12 01:23:09 on dist-test-slave-t3q3"
server_key: "63f55870b2670daf8e66de46877c504e"
server_key_iv: "345ce9998386c3fb8c41004fd128ac4c"
server_key_version: "encryptionkey@0"
I20260812 01:23:09.438882 26297 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 01:23:09.439596 26320 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:23:09.439775 26297 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 01:23:09.439883 26297 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786497534195564-24578-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786497534195564-24578-0/minicluster-data/master-0/wal
uuid: "e550a01aaebf424a820d877f587c8ce5"
format_stamp: "Formatted at 2026-08-12 01:23:09 on dist-test-slave-t3q3"
server_key: "63f55870b2670daf8e66de46877c504e"
server_key_iv: "345ce9998386c3fb8c41004fd128ac4c"
server_key_version: "encryptionkey@0"
I20260812 01:23:09.439960 26297 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786497534195564-24578-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786497534195564-24578-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786497534195564-24578-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:23:09.465551 26297 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:23:09.468029 26297 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:23:09.468228 26297 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:23:09.472280 26297 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45793
I20260812 01:23:09.472326 26401 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45793 every 8 connection(s)
I20260812 01:23:09.472731 26297 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
I20260812 01:23:09.473625 26402 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:23:09.475854 26402 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e550a01aaebf424a820d877f587c8ce5: Bootstrap starting.
I20260812 01:23:09.476431 26402 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e550a01aaebf424a820d877f587c8ce5: Neither blocks nor log segments found. Creating new log.
I20260812 01:23:09.476672 26402 log.cc:824] T 00000000000000000000000000000000 P e550a01aaebf424a820d877f587c8ce5: Log is configured to *not* fsync() on all Append() calls
I20260812 01:23:09.477346 26402 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e550a01aaebf424a820d877f587c8ce5: No bootstrap required, opened a new log
I20260812 01:23:09.479008 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 26297
I20260812 01:23:09.478973 26402 raft_consensus.cc:348] T 00000000000000000000000000000000 P e550a01aaebf424a820d877f587c8ce5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e550a01aaebf424a820d877f587c8ce5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45793 } }
I20260812 01:23:09.479137 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786497534195564-24578-0/minicluster-data/master-0/wal/instance
I20260812 01:23:09.479172 26402 raft_consensus.cc:374] T 00000000000000000000000000000000 P e550a01aaebf424a820d877f587c8ce5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:23:09.479223 26402 raft_consensus.cc:729] T 00000000000000000000000000000000 P e550a01aaebf424a820d877f587c8ce5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e550a01aaebf424a820d877f587c8ce5, State: Initialized, Role: FOLLOWER
I20260812 01:23:09.479349 24578 external_mini_cluster.cc:1428] Setting key 49df725a984d2785a44cf46cad567a64
I20260812 01:23:09.479383 26402 consensus_queue.cc:260] T 00000000000000000000000000000000 P e550a01aaebf424a820d877f587c8ce5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e550a01aaebf424a820d877f587c8ce5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45793 } }
I20260812 01:23:09.479473 26402 raft_consensus.cc:388] T 00000000000000000000000000000000 P e550a01aaebf424a820d877f587c8ce5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:23:09.479511 26402 raft_consensus.cc:482] T 00000000000000000000000000000000 P e550a01aaebf424a820d877f587c8ce5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:23:09.479568 26402 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e550a01aaebf424a820d877f587c8ce5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:23:09.480684 26402 raft_consensus.cc:504] T 00000000000000000000000000000000 P e550a01aaebf424a820d877f587c8ce5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e550a01aaebf424a820d877f587c8ce5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45793 } }
I20260812 01:23:09.480778 26402 leader_election.cc:302] T 00000000000000000000000000000000 P e550a01aaebf424a820d877f587c8ce5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e550a01aaebf424a820d877f587c8ce5; no voters:
I20260812 01:23:09.480978 26402 leader_election.cc:288] T 00000000000000000000000000000000 P e550a01aaebf424a820d877f587c8ce5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 01:23:09.481036 26411 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e550a01aaebf424a820d877f587c8ce5 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:23:09.481190 26411 raft_consensus.cc:686] T 00000000000000000000000000000000 P e550a01aaebf424a820d877f587c8ce5 [term 1 LEADER]: Becoming Leader. State: Replica: e550a01aaebf424a820d877f587c8ce5, State: Running, Role: LEADER
I20260812 01:23:09.481268 26402 sys_catalog.cc:564] T 00000000000000000000000000000000 P e550a01aaebf424a820d877f587c8ce5 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:23:09.481345 26411 consensus_queue.cc:237] T 00000000000000000000000000000000 P e550a01aaebf424a820d877f587c8ce5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e550a01aaebf424a820d877f587c8ce5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45793 } }
I20260812 01:23:09.481828 26413 sys_catalog.cc:455] T 00000000000000000000000000000000 P e550a01aaebf424a820d877f587c8ce5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e550a01aaebf424a820d877f587c8ce5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e550a01aaebf424a820d877f587c8ce5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45793 } } }
I20260812 01:23:09.481901 26413 sys_catalog.cc:458] T 00000000000000000000000000000000 P e550a01aaebf424a820d877f587c8ce5 [sys.catalog]: This master's current role is: LEADER
I20260812 01:23:09.482095 26415 sys_catalog.cc:455] T 00000000000000000000000000000000 P e550a01aaebf424a820d877f587c8ce5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e550a01aaebf424a820d877f587c8ce5. Latest consensus state: current_term: 1 leader_uuid: "e550a01aaebf424a820d877f587c8ce5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e550a01aaebf424a820d877f587c8ce5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45793 } } }
I20260812 01:23:09.482197 26415 sys_catalog.cc:458] T 00000000000000000000000000000000 P e550a01aaebf424a820d877f587c8ce5 [sys.catalog]: This master's current role is: LEADER
I20260812 01:23:09.482455 26430 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:23:09.483024 26430 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:23:09.485044 26430 catalog_manager.cc:1347] Generated new cluster ID: f87973edc7494de7b9a7ea8a8838eaaf
I20260812 01:23:09.485105 26430 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 01:23:09 dist-test-slave-t3q3 krb5kdc[26278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497789, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 01:23:09.491093 26409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:09.479813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34803 (local address 127.0.0.1:45793)
0812 01:23:09.480048 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:09.480083 (+ 35us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:09.480121 (+ 38us) server_negotiation.cc:407] Connection header received
0812 01:23:09.480271 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:09.480281 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:09.480349 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:09.480506 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:23:09.481294 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:09.482690 (+ 1396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:09.483694 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:09.484300 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:09.484417 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:09.487249 (+ 2832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:23:09.487253 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:23:09.487253 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:23:09.487265 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:23:09.489298 (+ 2033us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:23:09.489706 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:23:09.489710 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:23:09.489711 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:23:09.489852 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:23:09.490065 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:23:09.490066 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:23:09.490066 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:23:09.490374 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:23:09.490534 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:09.490635 (+ 101us) server_negotiation.cc:299] Negotiation successful
0812 01:23:09.490895 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":113,"threads_started":1}
I20260812 01:23:09.492352 24578 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 01:23:09.503203 26409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:09.493838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55735 (local address 127.0.0.1:45793)
0812 01:23:09.493871 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:09.493873 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:09.496265 (+ 2392us) server_negotiation.cc:407] Connection header received
0812 01:23:09.496299 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:09.496300 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:09.496311 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:09.496377 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:23:09.496955 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:09.497701 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:09.498642 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:09.499009 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:09.499083 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:09.500432 (+ 1349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:23:09.500434 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:23:09.500434 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:23:09.500441 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:23:09.502045 (+ 1604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:23:09.502420 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:23:09.502421 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:23:09.502421 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:23:09.502483 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:23:09.502688 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:23:09.502689 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:23:09.502689 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:23:09.502821 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:23:09.502931 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:09.503027 (+ 96us) server_negotiation.cc:299] Negotiation successful
0812 01:23:09.503104 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260812 01:23:09.505445 26430 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:23:09.506207 26430 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:23:09.512265 26430 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e550a01aaebf424a820d877f587c8ce5: Generated new TSK 0
I20260812 01:23:09.512449 26430 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 01:23:09.518246 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 26297
2026-08-12T01:23:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1584 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786497534195564-24578-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 01:23:09 dist-test-slave-t3q3 krb5kdc[26453](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 01:23:09 dist-test-slave-t3q3 krb5kdc[26453](info): set up 2 sockets
Aug 12 01:23:09 dist-test-slave-t3q3 krb5kdc[26453](info): commencing operation
krb5kdc: starting...
W20260812 01:23:10.795609 24578 mini_kdc.cc:121] Time spent starting KDC: real 1.242s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786497534195564-24578-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786497534195564-24578-0/krb5kdc/test-user.keytab.
Aug 12 01:23:10 dist-test-slave-t3q3 krb5kdc[26453](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497790, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T01:23:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:23:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786497534195564-24578-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786497534195564-24578-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 01:23:10.927474 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786497534195564-24578-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786497534195564-24578-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786497534195564-24578-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33303
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786497534195564-24578-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33303
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:23:11.011121 26472 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:23:11.011397 26472 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:23:11.011438 26472 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:23:11.012627 26472 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:23:11.012705 26472 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:23:11.012732 26472 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:23:11.012763 26472 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:23:11.012792 26472 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:23:11.014313 26472 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786497534195564-24578-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786497534195564-24578-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33303
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786497534195564-24578-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33303
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26472
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786497534195564-24578-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:23:11.014605 26472 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:23:11.014853 26472 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:23:11.019923 26484 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:23:11.019922 26490 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:23:11.019929 26481 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:23:11.020036 26472 server_base.cc:1034] running on GCE node
I20260812 01:23:11.020466 26472 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:23:11.020780 26472 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:23:11.021930 26472 hybrid_clock.cc:648] HybridClock initialized: now 1786497791021901 us; error 44 us; skew 500 ppm
Aug 12 01:23:11 dist-test-slave-t3q3 krb5kdc[26453](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497791, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:23:11.024442 26472 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 01:23:11.024928 26472 webserver.cc:460] Webserver started at http://127.0.0.1:37009/ using document root <none> and password file <none>
I20260812 01:23:11.025190 26472 fs_manager.cc:359] Metadata directory not provided
I20260812 01:23:11.025236 26472 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:23:11.025413 26472 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:23:11.026492 26472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786497534195564-24578-0/minicluster-data/master-0/data/instance:
uuid: "ecd829410c3248cdba2eff24d7f7644f"
format_stamp: "Formatted at 2026-08-12 01:23:11 on dist-test-slave-t3q3"
server_key: "d4d34b15544dc31b6c2967edc0e4cf54"
server_key_iv: "f0d474bd35a9a3982bdd064aa53ae991"
server_key_version: "encryptionkey@0"
I20260812 01:23:11.026867 26472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786497534195564-24578-0/minicluster-data/master-0/wal/instance:
uuid: "ecd829410c3248cdba2eff24d7f7644f"
format_stamp: "Formatted at 2026-08-12 01:23:11 on dist-test-slave-t3q3"
server_key: "d4d34b15544dc31b6c2967edc0e4cf54"
server_key_iv: "f0d474bd35a9a3982bdd064aa53ae991"
server_key_version: "encryptionkey@0"
I20260812 01:23:11.028264 26472 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 01:23:11.029007 26498 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:23:11.029305 26472 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:23:11.029412 26472 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786497534195564-24578-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786497534195564-24578-0/minicluster-data/master-0/wal
uuid: "ecd829410c3248cdba2eff24d7f7644f"
format_stamp: "Formatted at 2026-08-12 01:23:11 on dist-test-slave-t3q3"
server_key: "d4d34b15544dc31b6c2967edc0e4cf54"
server_key_iv: "f0d474bd35a9a3982bdd064aa53ae991"
server_key_version: "encryptionkey@0"
I20260812 01:23:11.029484 26472 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786497534195564-24578-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786497534195564-24578-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786497534195564-24578-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:23:11.039469 26472 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:23:11.041597 26472 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:23:11.041749 26472 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:23:11.046151 26472 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33303
I20260812 01:23:11.046212 26578 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33303 every 8 connection(s)
I20260812 01:23:11.046571 26472 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
I20260812 01:23:11.047529 26581 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:23:11.049944 26581 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ecd829410c3248cdba2eff24d7f7644f: Bootstrap starting.
I20260812 01:23:11.050509 26581 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ecd829410c3248cdba2eff24d7f7644f: Neither blocks nor log segments found. Creating new log.
I20260812 01:23:11.050769 26581 log.cc:824] T 00000000000000000000000000000000 P ecd829410c3248cdba2eff24d7f7644f: Log is configured to *not* fsync() on all Append() calls
I20260812 01:23:11.051517 26581 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ecd829410c3248cdba2eff24d7f7644f: No bootstrap required, opened a new log
I20260812 01:23:11.053099 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 26472
I20260812 01:23:11.053232 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786497534195564-24578-0/minicluster-data/master-0/wal/instance
I20260812 01:23:11.053426 24578 external_mini_cluster.cc:1428] Setting key fef9613f7e67e93146034dc7eacee57e
I20260812 01:23:11.053529 26581 raft_consensus.cc:348] T 00000000000000000000000000000000 P ecd829410c3248cdba2eff24d7f7644f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecd829410c3248cdba2eff24d7f7644f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33303 } }
I20260812 01:23:11.053730 26581 raft_consensus.cc:374] T 00000000000000000000000000000000 P ecd829410c3248cdba2eff24d7f7644f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:23:11.053781 26581 raft_consensus.cc:729] T 00000000000000000000000000000000 P ecd829410c3248cdba2eff24d7f7644f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ecd829410c3248cdba2eff24d7f7644f, State: Initialized, Role: FOLLOWER
I20260812 01:23:11.053910 26581 consensus_queue.cc:260] T 00000000000000000000000000000000 P ecd829410c3248cdba2eff24d7f7644f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecd829410c3248cdba2eff24d7f7644f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33303 } }
I20260812 01:23:11.054054 26581 raft_consensus.cc:388] T 00000000000000000000000000000000 P ecd829410c3248cdba2eff24d7f7644f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:23:11.054144 26581 raft_consensus.cc:482] T 00000000000000000000000000000000 P ecd829410c3248cdba2eff24d7f7644f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:23:11.054239 26581 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ecd829410c3248cdba2eff24d7f7644f [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:23:11.055163 26581 raft_consensus.cc:504] T 00000000000000000000000000000000 P ecd829410c3248cdba2eff24d7f7644f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecd829410c3248cdba2eff24d7f7644f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33303 } }
I20260812 01:23:11.055292 26581 leader_election.cc:302] T 00000000000000000000000000000000 P ecd829410c3248cdba2eff24d7f7644f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ecd829410c3248cdba2eff24d7f7644f; no voters:
I20260812 01:23:11.055496 26581 leader_election.cc:288] T 00000000000000000000000000000000 P ecd829410c3248cdba2eff24d7f7644f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 01:23:11.055534 26587 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ecd829410c3248cdba2eff24d7f7644f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:23:11.055644 26587 raft_consensus.cc:686] T 00000000000000000000000000000000 P ecd829410c3248cdba2eff24d7f7644f [term 1 LEADER]: Becoming Leader. State: Replica: ecd829410c3248cdba2eff24d7f7644f, State: Running, Role: LEADER
I20260812 01:23:11.055781 26581 sys_catalog.cc:564] T 00000000000000000000000000000000 P ecd829410c3248cdba2eff24d7f7644f [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:23:11.055792 26587 consensus_queue.cc:237] T 00000000000000000000000000000000 P ecd829410c3248cdba2eff24d7f7644f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecd829410c3248cdba2eff24d7f7644f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33303 } }
I20260812 01:23:11.057116 26589 sys_catalog.cc:455] T 00000000000000000000000000000000 P ecd829410c3248cdba2eff24d7f7644f [sys.catalog]: SysCatalogTable state changed. Reason: New leader ecd829410c3248cdba2eff24d7f7644f. Latest consensus state: current_term: 1 leader_uuid: "ecd829410c3248cdba2eff24d7f7644f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecd829410c3248cdba2eff24d7f7644f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33303 } } }
I20260812 01:23:11.057204 26589 sys_catalog.cc:458] T 00000000000000000000000000000000 P ecd829410c3248cdba2eff24d7f7644f [sys.catalog]: This master's current role is: LEADER
I20260812 01:23:11.057456 26588 sys_catalog.cc:455] T 00000000000000000000000000000000 P ecd829410c3248cdba2eff24d7f7644f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ecd829410c3248cdba2eff24d7f7644f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecd829410c3248cdba2eff24d7f7644f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33303 } } }
I20260812 01:23:11.057542 26588 sys_catalog.cc:458] T 00000000000000000000000000000000 P ecd829410c3248cdba2eff24d7f7644f [sys.catalog]: This master's current role is: LEADER
W20260812 01:23:11.057885 26619 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ecd829410c3248cdba2eff24d7f7644f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 01:23:11.057951 26619 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 01:23:11.058009 26620 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:23:11.058192 26620 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:23:11.059310 26586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:11.053962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35145 (local address 127.0.0.1:33303)
0812 01:23:11.054171 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:11.054182 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:11.054219 (+ 37us) server_negotiation.cc:407] Connection header received
0812 01:23:11.054349 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:11.054355 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:11.054408 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:11.054546 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:23:11.054763 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:11.055882 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:11.057019 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:11.057675 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:11.057798 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:11.058580 (+ 782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:23:11.058584 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:23:11.058585 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:23:11.058796 (+ 211us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:23:11.058838 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:23:11.058853 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:23:11.058916 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:11.058997 (+ 81us) server_negotiation.cc:299] Negotiation successful
0812 01:23:11.059062 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":99,"threads_started":1}
I20260812 01:23:11.059980 26620 catalog_manager.cc:1347] Generated new cluster ID: 12404a9ddd2248938ba44a020cf2067e
I20260812 01:23:11.060031 26620 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 01:23:11.060350 24578 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 01:23:11.065364 26586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:11.061720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43991 (local address 127.0.0.1:33303)
0812 01:23:11.061784 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:11.061785 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:11.062012 (+ 227us) server_negotiation.cc:407] Connection header received
0812 01:23:11.062040 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:11.062041 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:11.062053 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:11.062122 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:23:11.062240 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:11.062970 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:11.063925 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:11.064338 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:11.064425 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:11.065112 (+ 687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:23:11.065113 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:23:11.065114 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:23:11.065118 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:23:11.065136 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:23:11.065141 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:23:11.065202 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:11.065264 (+ 62us) server_negotiation.cc:299] Negotiation successful
0812 01:23:11.065290 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 01:23:11.068930 26620 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:23:11.069483 26620 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:23:11.076133 26620 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ecd829410c3248cdba2eff24d7f7644f: Generated new TSK 0
I20260812 01:23:11.076371 26620 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 01:23:11.086345 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 26472
2026-08-12T01:23:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1565 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786497534195564-24578-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 01:23:11 dist-test-slave-t3q3 krb5kdc[26630](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 01:23:11 dist-test-slave-t3q3 krb5kdc[26630](info): set up 2 sockets
Aug 12 01:23:11 dist-test-slave-t3q3 krb5kdc[26630](info): commencing operation
krb5kdc: starting...
W20260812 01:23:12.365918 24578 mini_kdc.cc:121] Time spent starting KDC: real 1.247s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786497534195564-24578-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786497534195564-24578-0/krb5kdc/test-user.keytab.
Aug 12 01:23:12 dist-test-slave-t3q3 krb5kdc[26630](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497792, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T01:23:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:23:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786497534195564-24578-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786497534195564-24578-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 01:23:12.489377 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786497534195564-24578-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786497534195564-24578-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786497534195564-24578-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39799
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786497534195564-24578-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39799
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:23:12.573338 26649 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:23:12.573531 26649 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:23:12.573555 26649 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:23:12.574754 26649 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:23:12.574796 26649 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:23:12.574810 26649 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:23:12.574851 26649 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:23:12.574872 26649 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:23:12.576395 26649 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786497534195564-24578-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786497534195564-24578-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39799
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786497534195564-24578-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39799
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26649
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786497534195564-24578-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:23:12.576618 26649 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:23:12.576834 26649 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:23:12.581693 26658 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:23:12.581671 26663 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:23:12.581682 26657 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:23:12.581787 26649 server_base.cc:1034] running on GCE node
I20260812 01:23:12.582183 26649 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:23:12.582470 26649 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:23:12.583634 26649 hybrid_clock.cc:648] HybridClock initialized: now 1786497792583615 us; error 49 us; skew 500 ppm
Aug 12 01:23:12 dist-test-slave-t3q3 krb5kdc[26630](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497792, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:23:12.586229 26649 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 01:23:12.586598 26649 webserver.cc:460] Webserver started at http://127.0.0.1:38627/ using document root <none> and password file <none>
I20260812 01:23:12.586781 26649 fs_manager.cc:359] Metadata directory not provided
I20260812 01:23:12.586819 26649 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:23:12.586907 26649 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:23:12.587875 26649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786497534195564-24578-0/minicluster-data/master-0/data/instance:
uuid: "a7ddee0e24aa47a6850ff699a18bc254"
format_stamp: "Formatted at 2026-08-12 01:23:12 on dist-test-slave-t3q3"
server_key: "ce9fbcdc2e2df498c065628585dc87ec"
server_key_iv: "debed3425b088c0e0cc29842210cfc65"
server_key_version: "encryptionkey@0"
I20260812 01:23:12.588248 26649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786497534195564-24578-0/minicluster-data/master-0/wal/instance:
uuid: "a7ddee0e24aa47a6850ff699a18bc254"
format_stamp: "Formatted at 2026-08-12 01:23:12 on dist-test-slave-t3q3"
server_key: "ce9fbcdc2e2df498c065628585dc87ec"
server_key_iv: "debed3425b088c0e0cc29842210cfc65"
server_key_version: "encryptionkey@0"
I20260812 01:23:12.589489 26649 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 01:23:12.590142 26678 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:23:12.590322 26649 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 01:23:12.590409 26649 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786497534195564-24578-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786497534195564-24578-0/minicluster-data/master-0/wal
uuid: "a7ddee0e24aa47a6850ff699a18bc254"
format_stamp: "Formatted at 2026-08-12 01:23:12 on dist-test-slave-t3q3"
server_key: "ce9fbcdc2e2df498c065628585dc87ec"
server_key_iv: "debed3425b088c0e0cc29842210cfc65"
server_key_version: "encryptionkey@0"
I20260812 01:23:12.590473 26649 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786497534195564-24578-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786497534195564-24578-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786497534195564-24578-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:23:12.593725 26649 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:23:12.595589 26649 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:23:12.595742 26649 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:23:12.599687 26649 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39799
I20260812 01:23:12.599720 26773 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39799 every 8 connection(s)
I20260812 01:23:12.600035 26649 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
I20260812 01:23:12.601027 26774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:23:12.603130 26774 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7ddee0e24aa47a6850ff699a18bc254: Bootstrap starting.
I20260812 01:23:12.603694 26774 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a7ddee0e24aa47a6850ff699a18bc254: Neither blocks nor log segments found. Creating new log.
I20260812 01:23:12.603978 26774 log.cc:824] T 00000000000000000000000000000000 P a7ddee0e24aa47a6850ff699a18bc254: Log is configured to *not* fsync() on all Append() calls
I20260812 01:23:12.604687 26774 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7ddee0e24aa47a6850ff699a18bc254: No bootstrap required, opened a new log
I20260812 01:23:12.605033 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 26649
I20260812 01:23:12.605154 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786497534195564-24578-0/minicluster-data/master-0/wal/instance
I20260812 01:23:12.605340 24578 external_mini_cluster.cc:1428] Setting key e4b596f60407deb2ea4f48afaff6adc6
I20260812 01:23:12.606997 26774 raft_consensus.cc:348] T 00000000000000000000000000000000 P a7ddee0e24aa47a6850ff699a18bc254 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7ddee0e24aa47a6850ff699a18bc254" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39799 } }
I20260812 01:23:12.607187 26774 raft_consensus.cc:374] T 00000000000000000000000000000000 P a7ddee0e24aa47a6850ff699a18bc254 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:23:12.607228 26774 raft_consensus.cc:729] T 00000000000000000000000000000000 P a7ddee0e24aa47a6850ff699a18bc254 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7ddee0e24aa47a6850ff699a18bc254, State: Initialized, Role: FOLLOWER
I20260812 01:23:12.607432 26774 consensus_queue.cc:260] T 00000000000000000000000000000000 P a7ddee0e24aa47a6850ff699a18bc254 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7ddee0e24aa47a6850ff699a18bc254" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39799 } }
I20260812 01:23:12.607555 26774 raft_consensus.cc:388] T 00000000000000000000000000000000 P a7ddee0e24aa47a6850ff699a18bc254 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:23:12.607604 26774 raft_consensus.cc:482] T 00000000000000000000000000000000 P a7ddee0e24aa47a6850ff699a18bc254 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:23:12.607661 26774 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a7ddee0e24aa47a6850ff699a18bc254 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:23:12.608412 26780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:12.605844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35763 (local address 127.0.0.1:39799)
0812 01:23:12.606084 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:12.606093 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:12.606127 (+ 34us) server_negotiation.cc:407] Connection header received
0812 01:23:12.606952 (+ 825us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:12.606962 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:12.607000 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:12.607156 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:23:12.607847 (+ 691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:23:12.607851 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:23:12.607852 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:23:12.607857 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:23:12.607902 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:23:12.607916 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:23:12.607994 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:12.608054 (+ 60us) server_negotiation.cc:299] Negotiation successful
0812 01:23:12.608169 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":106,"threads_started":1}
I20260812 01:23:12.608963 26774 raft_consensus.cc:504] T 00000000000000000000000000000000 P a7ddee0e24aa47a6850ff699a18bc254 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7ddee0e24aa47a6850ff699a18bc254" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39799 } }
I20260812 01:23:12.609091 26774 leader_election.cc:302] T 00000000000000000000000000000000 P a7ddee0e24aa47a6850ff699a18bc254 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a7ddee0e24aa47a6850ff699a18bc254; no voters:
I20260812 01:23:12.609280 26774 leader_election.cc:288] T 00000000000000000000000000000000 P a7ddee0e24aa47a6850ff699a18bc254 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 01:23:12.609349 26782 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a7ddee0e24aa47a6850ff699a18bc254 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:23:12.609478 24578 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 01:23:12.609529 26774 sys_catalog.cc:564] T 00000000000000000000000000000000 P a7ddee0e24aa47a6850ff699a18bc254 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:23:12.610867 26782 raft_consensus.cc:686] T 00000000000000000000000000000000 P a7ddee0e24aa47a6850ff699a18bc254 [term 1 LEADER]: Becoming Leader. State: Replica: a7ddee0e24aa47a6850ff699a18bc254, State: Running, Role: LEADER
I20260812 01:23:12.610991 26782 consensus_queue.cc:237] T 00000000000000000000000000000000 P a7ddee0e24aa47a6850ff699a18bc254 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7ddee0e24aa47a6850ff699a18bc254" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39799 } }
I20260812 01:23:12.611430 26810 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7ddee0e24aa47a6850ff699a18bc254 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a7ddee0e24aa47a6850ff699a18bc254. Latest consensus state: current_term: 1 leader_uuid: "a7ddee0e24aa47a6850ff699a18bc254" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7ddee0e24aa47a6850ff699a18bc254" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39799 } } }
I20260812 01:23:12.611529 26810 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7ddee0e24aa47a6850ff699a18bc254 [sys.catalog]: This master's current role is: LEADER
I20260812 01:23:12.611539 26786 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7ddee0e24aa47a6850ff699a18bc254 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a7ddee0e24aa47a6850ff699a18bc254" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7ddee0e24aa47a6850ff699a18bc254" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39799 } } }
I20260812 01:23:12.611927 26786 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7ddee0e24aa47a6850ff699a18bc254 [sys.catalog]: This master's current role is: LEADER
I20260812 01:23:12.611979 26815 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:23:12.612573 26815 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:23:12.613456 26780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:12.611517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39319 (local address 127.0.0.1:39799)
0812 01:23:12.611554 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:12.611556 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:12.612464 (+ 908us) server_negotiation.cc:407] Connection header received
0812 01:23:12.612519 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:12.612520 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:12.612525 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:12.612585 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:23:12.613256 (+ 671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:23:12.613257 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:23:12.613258 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:23:12.613262 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:23:12.613274 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:23:12.613278 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:23:12.613324 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:12.613378 (+ 54us) server_negotiation.cc:299] Negotiation successful
0812 01:23:12.613392 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 01:23:12.614298 26815 catalog_manager.cc:1347] Generated new cluster ID: 1e72ae866fee4cd88c75ac34a7c85c1e
I20260812 01:23:12.614360 26815 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 01:23:12.627169 26815 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:23:12.627358 26815 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:23:12.636780 26815 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a7ddee0e24aa47a6850ff699a18bc254: Generated new TSK 0
I20260812 01:23:12.637030 26815 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 01:23:12.645949 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 26649
2026-08-12T01:23:12Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1556 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786497534195564-24578-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 01:23:12 dist-test-slave-t3q3 krb5kdc[26834](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 01:23:12 dist-test-slave-t3q3 krb5kdc[26834](info): set up 2 sockets
Aug 12 01:23:12 dist-test-slave-t3q3 krb5kdc[26834](info): commencing operation
krb5kdc: starting...
W20260812 01:23:13.916450 24578 mini_kdc.cc:121] Time spent starting KDC: real 1.241s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786497534195564-24578-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786497534195564-24578-0/krb5kdc/test-user.keytab.
Aug 12 01:23:14 dist-test-slave-t3q3 krb5kdc[26834](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497794, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T01:23:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:23:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786497534195564-24578-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786497534195564-24578-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 01:23:14.070604 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786497534195564-24578-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786497534195564-24578-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786497534195564-24578-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35059
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786497534195564-24578-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:32851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35059
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:23:14.154136 26858 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:23:14.154415 26858 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:23:14.154459 26858 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:23:14.155635 26858 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:23:14.155711 26858 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:23:14.155737 26858 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:23:14.155763 26858 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:23:14.155795 26858 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:23:14.157316 26858 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:32851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786497534195564-24578-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786497534195564-24578-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35059
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786497534195564-24578-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35059
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26858
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786497534195564-24578-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:23:14.157601 26858 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:23:14.157845 26858 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:23:14.162926 26864 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:23:14.162921 26867 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:23:14.162993 26863 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:23:14.163168 26858 server_base.cc:1034] running on GCE node
I20260812 01:23:14.163424 26858 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:23:14.163765 26858 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:23:14.164932 26858 hybrid_clock.cc:648] HybridClock initialized: now 1786497794164925 us; error 36 us; skew 500 ppm
Aug 12 01:23:14 dist-test-slave-t3q3 krb5kdc[26834](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497794, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:23:14.167482 26858 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 01:23:14.167884 26858 webserver.cc:460] Webserver started at http://127.0.0.1:35141/ using document root <none> and password file <none>
I20260812 01:23:14.168128 26858 fs_manager.cc:359] Metadata directory not provided
I20260812 01:23:14.168190 26858 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:23:14.168284 26858 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:23:14.169317 26858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786497534195564-24578-0/minicluster-data/master-0/data/instance:
uuid: "5d1175034f314c5d829d6e0ef4d12cc2"
format_stamp: "Formatted at 2026-08-12 01:23:14 on dist-test-slave-t3q3"
server_key: "2b9d5cb881e67955f9f26d22a5536e65"
server_key_iv: "14d96ae51e641c7e80598433c4c14204"
server_key_version: "encryptionkey@0"
I20260812 01:23:14.169622 26858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786497534195564-24578-0/minicluster-data/master-0/wal/instance:
uuid: "5d1175034f314c5d829d6e0ef4d12cc2"
format_stamp: "Formatted at 2026-08-12 01:23:14 on dist-test-slave-t3q3"
server_key: "2b9d5cb881e67955f9f26d22a5536e65"
server_key_iv: "14d96ae51e641c7e80598433c4c14204"
server_key_version: "encryptionkey@0"
I20260812 01:23:14.170908 26858 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 01:23:14.171583 26881 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:23:14.171757 26858 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 01:23:14.171851 26858 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786497534195564-24578-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786497534195564-24578-0/minicluster-data/master-0/wal
uuid: "5d1175034f314c5d829d6e0ef4d12cc2"
format_stamp: "Formatted at 2026-08-12 01:23:14 on dist-test-slave-t3q3"
server_key: "2b9d5cb881e67955f9f26d22a5536e65"
server_key_iv: "14d96ae51e641c7e80598433c4c14204"
server_key_version: "encryptionkey@0"
I20260812 01:23:14.171907 26858 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786497534195564-24578-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786497534195564-24578-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786497534195564-24578-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:23:14.195003 26858 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:23:14.197561 26858 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:23:14.197685 26858 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:23:14.201813 26858 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35059
I20260812 01:23:14.201897 26959 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35059 every 8 connection(s)
I20260812 01:23:14.202286 26858 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
I20260812 01:23:14.203256 26960 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:23:14.205744 26960 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d1175034f314c5d829d6e0ef4d12cc2: Bootstrap starting.
I20260812 01:23:14.206106 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 26858
I20260812 01:23:14.206231 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786497534195564-24578-0/minicluster-data/master-0/wal/instance
I20260812 01:23:14.206429 24578 external_mini_cluster.cc:1428] Setting key 01b77692abcc537fd3d847088f79444f
I20260812 01:23:14.206447 26960 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5d1175034f314c5d829d6e0ef4d12cc2: Neither blocks nor log segments found. Creating new log.
I20260812 01:23:14.206832 26960 log.cc:824] T 00000000000000000000000000000000 P 5d1175034f314c5d829d6e0ef4d12cc2: Log is configured to *not* fsync() on all Append() calls
I20260812 01:23:14.207845 26960 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d1175034f314c5d829d6e0ef4d12cc2: No bootstrap required, opened a new log
I20260812 01:23:14.210326 26960 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5d1175034f314c5d829d6e0ef4d12cc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d1175034f314c5d829d6e0ef4d12cc2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35059 } }
I20260812 01:23:14.210503 26960 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5d1175034f314c5d829d6e0ef4d12cc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:23:14.210541 26960 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5d1175034f314c5d829d6e0ef4d12cc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d1175034f314c5d829d6e0ef4d12cc2, State: Initialized, Role: FOLLOWER
I20260812 01:23:14.210667 26960 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5d1175034f314c5d829d6e0ef4d12cc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d1175034f314c5d829d6e0ef4d12cc2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35059 } }
I20260812 01:23:14.210741 26960 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5d1175034f314c5d829d6e0ef4d12cc2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:23:14.210769 26960 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5d1175034f314c5d829d6e0ef4d12cc2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:23:14.210813 26960 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5d1175034f314c5d829d6e0ef4d12cc2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:23:14.211448 26960 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5d1175034f314c5d829d6e0ef4d12cc2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d1175034f314c5d829d6e0ef4d12cc2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35059 } }
I20260812 01:23:14.211541 26960 leader_election.cc:302] T 00000000000000000000000000000000 P 5d1175034f314c5d829d6e0ef4d12cc2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5d1175034f314c5d829d6e0ef4d12cc2; no voters:
I20260812 01:23:14.211802 26960 leader_election.cc:288] T 00000000000000000000000000000000 P 5d1175034f314c5d829d6e0ef4d12cc2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 01:23:14.211822 26972 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5d1175034f314c5d829d6e0ef4d12cc2 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:23:14.211954 26972 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5d1175034f314c5d829d6e0ef4d12cc2 [term 1 LEADER]: Becoming Leader. State: Replica: 5d1175034f314c5d829d6e0ef4d12cc2, State: Running, Role: LEADER
I20260812 01:23:14.212126 26960 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5d1175034f314c5d829d6e0ef4d12cc2 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:23:14.212145 26972 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5d1175034f314c5d829d6e0ef4d12cc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d1175034f314c5d829d6e0ef4d12cc2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35059 } }
I20260812 01:23:14.212579 26972 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d1175034f314c5d829d6e0ef4d12cc2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5d1175034f314c5d829d6e0ef4d12cc2. Latest consensus state: current_term: 1 leader_uuid: "5d1175034f314c5d829d6e0ef4d12cc2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d1175034f314c5d829d6e0ef4d12cc2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35059 } } }
I20260812 01:23:14.212834 26972 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d1175034f314c5d829d6e0ef4d12cc2 [sys.catalog]: This master's current role is: LEADER
I20260812 01:23:14.212754 26979 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d1175034f314c5d829d6e0ef4d12cc2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5d1175034f314c5d829d6e0ef4d12cc2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d1175034f314c5d829d6e0ef4d12cc2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35059 } } }
I20260812 01:23:14.213279 26979 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d1175034f314c5d829d6e0ef4d12cc2 [sys.catalog]: This master's current role is: LEADER
I20260812 01:23:14.213493 26992 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 12 01:23:14 dist-test-slave-t3q3 krb5kdc[26834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497794, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 01:23:14.214164 26992 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:23:14.216048 26992 catalog_manager.cc:1347] Generated new cluster ID: d5d7c19318f4421d95725ce0bdadd050
I20260812 01:23:14.216126 26992 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 01:23:14.218168 26969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:14.207067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.50:33403 (local address 127.0.0.1:35059)
0812 01:23:14.207281 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:14.207292 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:14.207332 (+ 40us) server_negotiation.cc:407] Connection header received
0812 01:23:14.207875 (+ 543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:14.207884 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:14.207937 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:14.208177 (+ 240us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:23:14.209077 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:14.210066 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:14.211104 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:14.211540 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:14.211653 (+ 113us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:14.214621 (+ 2968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:23:14.214626 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:23:14.214626 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:23:14.214638 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:23:14.216625 (+ 1987us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:23:14.217019 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:23:14.217023 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:23:14.217023 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:23:14.217117 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:23:14.217362 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:23:14.217364 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:23:14.217364 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:23:14.217602 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:23:14.217798 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:14.217885 (+ 87us) server_negotiation.cc:299] Negotiation successful
0812 01:23:14.217985 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":79,"threads_started":1}
I20260812 01:23:14.221107 24578 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 01:23:14.222970 26992 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:23:14.223654 26992 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:23:14.230353 26969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:14.222508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.50:34649 (local address 127.0.0.1:35059)
0812 01:23:14.222547 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:14.222550 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:14.222886 (+ 336us) server_negotiation.cc:407] Connection header received
0812 01:23:14.224171 (+ 1285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:14.224172 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:14.224184 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:14.224220 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:23:14.224818 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:14.225430 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:14.226159 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:14.226384 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:14.226458 (+ 74us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:14.227704 (+ 1246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:23:14.227705 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:23:14.227706 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:23:14.227713 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:23:14.229167 (+ 1454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:23:14.229543 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:23:14.229544 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:23:14.229545 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:23:14.229605 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:23:14.229837 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:23:14.229838 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:23:14.229838 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:23:14.229948 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:23:14.230061 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:14.230177 (+ 116us) server_negotiation.cc:299] Negotiation successful
0812 01:23:14.230242 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 01:23:14.233228 26992 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5d1175034f314c5d829d6e0ef4d12cc2: Generated new TSK 0
I20260812 01:23:14.233404 26992 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 01:23:14.235879 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 26858
W20260812 01:23:14.253829 26830 connection.cc:535] client connection to 127.0.0.1:35059 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35059): Connection reset by peer (error 104)
2026-08-12T01:23:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1592 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786497534195564-24578-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 01:23:14 dist-test-slave-t3q3 krb5kdc[27015](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 01:23:14 dist-test-slave-t3q3 krb5kdc[27015](info): set up 2 sockets
Aug 12 01:23:14 dist-test-slave-t3q3 krb5kdc[27015](info): commencing operation
krb5kdc: starting...
W20260812 01:23:15.515439 24578 mini_kdc.cc:121] Time spent starting KDC: real 1.248s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786497534195564-24578-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786497534195564-24578-0/krb5kdc/test-user.keytab.
Aug 12 01:23:15 dist-test-slave-t3q3 krb5kdc[27015](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497795, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T01:23:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:23:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786497534195564-24578-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786497534195564-24578-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 01:23:15.637164 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786497534195564-24578-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786497534195564-24578-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786497534195564-24578-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43663
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786497534195564-24578-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43663
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:23:15.729511 27039 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:23:15.729739 27039 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:23:15.729765 27039 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:23:15.731068 27039 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:23:15.731117 27039 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:23:15.731132 27039 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:23:15.731199 27039 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:23:15.731220 27039 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:23:15.733011 27039 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786497534195564-24578-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786497534195564-24578-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43663
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786497534195564-24578-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43663
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27039
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786497534195564-24578-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:23:15.733304 27039 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:23:15.733569 27039 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 01:23:15.739037 27039 server_base.cc:1034] running on GCE node
W20260812 01:23:15.738986 27046 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:23:15.738998 27052 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:23:15.739081 27048 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:23:15.739452 27039 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:23:15.739774 27039 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:23:15.740942 27039 hybrid_clock.cc:648] HybridClock initialized: now 1786497795740936 us; error 37 us; skew 500 ppm
Aug 12 01:23:15 dist-test-slave-t3q3 krb5kdc[27015](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497795, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:23:15.747063 27039 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 01:23:15.747538 27039 webserver.cc:460] Webserver started at http://127.0.0.1:37559/ using document root <none> and password file <none>
I20260812 01:23:15.747747 27039 fs_manager.cc:359] Metadata directory not provided
I20260812 01:23:15.747793 27039 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:23:15.747900 27039 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:23:15.749140 27039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786497534195564-24578-0/minicluster-data/master-0/data/instance:
uuid: "cfc76f9fe6324160bbbec46b0b4428b9"
format_stamp: "Formatted at 2026-08-12 01:23:15 on dist-test-slave-t3q3"
server_key: "d1161797b1b48318b41b84b77a823585"
server_key_iv: "6042e8c438476d20b429c553d97f3ce4"
server_key_version: "encryptionkey@0"
I20260812 01:23:15.749507 27039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786497534195564-24578-0/minicluster-data/master-0/wal/instance:
uuid: "cfc76f9fe6324160bbbec46b0b4428b9"
format_stamp: "Formatted at 2026-08-12 01:23:15 on dist-test-slave-t3q3"
server_key: "d1161797b1b48318b41b84b77a823585"
server_key_iv: "6042e8c438476d20b429c553d97f3ce4"
server_key_version: "encryptionkey@0"
I20260812 01:23:15.751014 27039 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 01:23:15.751825 27069 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:23:15.752099 27039 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:23:15.752199 27039 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786497534195564-24578-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786497534195564-24578-0/minicluster-data/master-0/wal
uuid: "cfc76f9fe6324160bbbec46b0b4428b9"
format_stamp: "Formatted at 2026-08-12 01:23:15 on dist-test-slave-t3q3"
server_key: "d1161797b1b48318b41b84b77a823585"
server_key_iv: "6042e8c438476d20b429c553d97f3ce4"
server_key_version: "encryptionkey@0"
I20260812 01:23:15.752262 27039 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786497534195564-24578-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786497534195564-24578-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786497534195564-24578-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:23:15.765992 27039 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:23:15.768232 27039 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:23:15.768352 27039 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:23:15.772368 27039 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43663
I20260812 01:23:15.772426 27160 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43663 every 8 connection(s)
I20260812 01:23:15.772840 27039 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
I20260812 01:23:15.773782 27163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:23:15.776177 27163 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfc76f9fe6324160bbbec46b0b4428b9: Bootstrap starting.
I20260812 01:23:15.776775 27163 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cfc76f9fe6324160bbbec46b0b4428b9: Neither blocks nor log segments found. Creating new log.
I20260812 01:23:15.777083 27163 log.cc:824] T 00000000000000000000000000000000 P cfc76f9fe6324160bbbec46b0b4428b9: Log is configured to *not* fsync() on all Append() calls
I20260812 01:23:15.777817 27163 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfc76f9fe6324160bbbec46b0b4428b9: No bootstrap required, opened a new log
I20260812 01:23:15.779439 27163 raft_consensus.cc:348] T 00000000000000000000000000000000 P cfc76f9fe6324160bbbec46b0b4428b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfc76f9fe6324160bbbec46b0b4428b9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43663 } }
I20260812 01:23:15.779603 27163 raft_consensus.cc:374] T 00000000000000000000000000000000 P cfc76f9fe6324160bbbec46b0b4428b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:23:15.779646 27163 raft_consensus.cc:729] T 00000000000000000000000000000000 P cfc76f9fe6324160bbbec46b0b4428b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfc76f9fe6324160bbbec46b0b4428b9, State: Initialized, Role: FOLLOWER
I20260812 01:23:15.779819 27163 consensus_queue.cc:260] T 00000000000000000000000000000000 P cfc76f9fe6324160bbbec46b0b4428b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfc76f9fe6324160bbbec46b0b4428b9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43663 } }
I20260812 01:23:15.779906 27163 raft_consensus.cc:388] T 00000000000000000000000000000000 P cfc76f9fe6324160bbbec46b0b4428b9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:23:15.779945 27163 raft_consensus.cc:482] T 00000000000000000000000000000000 P cfc76f9fe6324160bbbec46b0b4428b9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:23:15.779999 27163 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cfc76f9fe6324160bbbec46b0b4428b9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:23:15.780576 27163 raft_consensus.cc:504] T 00000000000000000000000000000000 P cfc76f9fe6324160bbbec46b0b4428b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfc76f9fe6324160bbbec46b0b4428b9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43663 } }
I20260812 01:23:15.780699 27163 leader_election.cc:302] T 00000000000000000000000000000000 P cfc76f9fe6324160bbbec46b0b4428b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cfc76f9fe6324160bbbec46b0b4428b9; no voters:
I20260812 01:23:15.780881 27163 leader_election.cc:288] T 00000000000000000000000000000000 P cfc76f9fe6324160bbbec46b0b4428b9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 01:23:15.780929 27167 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cfc76f9fe6324160bbbec46b0b4428b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:23:15.781147 27167 raft_consensus.cc:686] T 00000000000000000000000000000000 P cfc76f9fe6324160bbbec46b0b4428b9 [term 1 LEADER]: Becoming Leader. State: Replica: cfc76f9fe6324160bbbec46b0b4428b9, State: Running, Role: LEADER
I20260812 01:23:15.781157 27163 sys_catalog.cc:564] T 00000000000000000000000000000000 P cfc76f9fe6324160bbbec46b0b4428b9 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:23:15.781349 27167 consensus_queue.cc:237] T 00000000000000000000000000000000 P cfc76f9fe6324160bbbec46b0b4428b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfc76f9fe6324160bbbec46b0b4428b9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43663 } }
I20260812 01:23:15.781736 27170 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfc76f9fe6324160bbbec46b0b4428b9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cfc76f9fe6324160bbbec46b0b4428b9. Latest consensus state: current_term: 1 leader_uuid: "cfc76f9fe6324160bbbec46b0b4428b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfc76f9fe6324160bbbec46b0b4428b9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43663 } } }
I20260812 01:23:15.781847 27170 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfc76f9fe6324160bbbec46b0b4428b9 [sys.catalog]: This master's current role is: LEADER
I20260812 01:23:15.781733 27168 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfc76f9fe6324160bbbec46b0b4428b9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cfc76f9fe6324160bbbec46b0b4428b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfc76f9fe6324160bbbec46b0b4428b9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43663 } } }
I20260812 01:23:15.782088 27172 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:23:15.782131 27168 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfc76f9fe6324160bbbec46b0b4428b9 [sys.catalog]: This master's current role is: LEADER
I20260812 01:23:15.782646 27172 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:23:15.782932 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 27039
I20260812 01:23:15.783056 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786497534195564-24578-0/minicluster-data/master-0/wal/instance
I20260812 01:23:15.783236 24578 external_mini_cluster.cc:1428] Setting key fb3c3dbd9b9ea9329e31ae9d50a81faf
I20260812 01:23:15.784435 27172 catalog_manager.cc:1347] Generated new cluster ID: 1c59a9bad646414b945f499460624dc5
I20260812 01:23:15.784493 27172 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 01:23:15.789783 27200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:15.783718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.50:37281 (local address 127.0.0.1:43663)
0812 01:23:15.784217 (+ 499us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:15.784227 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:15.784256 (+ 29us) server_negotiation.cc:407] Connection header received
0812 01:23:15.784368 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:15.784375 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:15.784435 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:15.784610 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:23:15.784865 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:15.786272 (+ 1407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:15.787434 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:15.788013 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:15.788131 (+ 118us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:15.788939 (+ 808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:23:15.788943 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:23:15.788943 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:23:15.789147 (+ 204us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:23:15.789196 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:23:15.789210 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:23:15.789290 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:15.789446 (+ 156us) server_negotiation.cc:299] Negotiation successful
0812 01:23:15.789573 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":451,"thread_start_us":78,"threads_started":1}
I20260812 01:23:15.790731 24578 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 01:23:15.795743 27200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:15.792196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.50:45861 (local address 127.0.0.1:43663)
0812 01:23:15.792239 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:15.792240 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:15.792350 (+ 110us) server_negotiation.cc:407] Connection header received
0812 01:23:15.792431 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:15.792431 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:15.792440 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:15.792466 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:23:15.792644 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:15.793397 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:15.794359 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:15.794676 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:15.794711 (+ 35us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:15.795466 (+ 755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:23:15.795467 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:23:15.795468 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:23:15.795472 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:23:15.795488 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:23:15.795493 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:23:15.795560 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:15.795656 (+ 96us) server_negotiation.cc:299] Negotiation successful
0812 01:23:15.795669 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 01:23:15.797693 27172 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:23:15.798180 27172 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:23:15.801915 27172 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cfc76f9fe6324160bbbec46b0b4428b9: Generated new TSK 0
I20260812 01:23:15.802111 27172 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 01:23:15.807121 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 27039
W20260812 01:23:15.830345 27013 connection.cc:535] client connection to 127.0.0.1:43663 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43663): Connection reset by peer (error 104)
2026-08-12T01:23:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1576 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786497534195564-24578-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 01:23:15 dist-test-slave-t3q3 krb5kdc[27215](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 01:23:15 dist-test-slave-t3q3 krb5kdc[27215](info): set up 2 sockets
Aug 12 01:23:15 dist-test-slave-t3q3 krb5kdc[27215](info): commencing operation
krb5kdc: starting...
W20260812 01:23:17.092370 24578 mini_kdc.cc:121] Time spent starting KDC: real 1.248s user 0.006s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786497534195564-24578-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786497534195564-24578-0/krb5kdc/test-user.keytab.
Aug 12 01:23:17 dist-test-slave-t3q3 krb5kdc[27215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497797, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T01:23:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:23:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786497534195564-24578-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786497534195564-24578-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 01:23:17.218923 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786497534195564-24578-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786497534195564-24578-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786497534195564-24578-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37159
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786497534195564-24578-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37159
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:23:17.302608 27235 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:23:17.302836 27235 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:23:17.302860 27235 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:23:17.304138 27235 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:23:17.304181 27235 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:23:17.304195 27235 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:23:17.304243 27235 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:23:17.304273 27235 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:23:17.306008 27235 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786497534195564-24578-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786497534195564-24578-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37159
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786497534195564-24578-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37159
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27235
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786497534195564-24578-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:23:17.306305 27235 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:23:17.306560 27235 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:23:17.311445 27244 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:23:17.311455 27248 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:23:17.311461 27243 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:23:17.311511 27235 server_base.cc:1034] running on GCE node
I20260812 01:23:17.311906 27235 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:23:17.312242 27235 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:23:17.313412 27235 hybrid_clock.cc:648] HybridClock initialized: now 1786497797313386 us; error 41 us; skew 500 ppm
Aug 12 01:23:17 dist-test-slave-t3q3 krb5kdc[27215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497797, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:23:17.316294 27235 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 01:23:17.316798 27235 webserver.cc:460] Webserver started at http://127.0.0.1:41537/ using document root <none> and password file <none>
I20260812 01:23:17.316998 27235 fs_manager.cc:359] Metadata directory not provided
I20260812 01:23:17.317041 27235 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:23:17.317129 27235 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:23:17.318114 27235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786497534195564-24578-0/minicluster-data/master-0/data/instance:
uuid: "aad578a4790d4377909bdd357bee7343"
format_stamp: "Formatted at 2026-08-12 01:23:17 on dist-test-slave-t3q3"
server_key: "677fd9664771bce405703bab0d3a497c"
server_key_iv: "81a351f81c8962b0ddb501b909c6c985"
server_key_version: "encryptionkey@0"
I20260812 01:23:17.318421 27235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786497534195564-24578-0/minicluster-data/master-0/wal/instance:
uuid: "aad578a4790d4377909bdd357bee7343"
format_stamp: "Formatted at 2026-08-12 01:23:17 on dist-test-slave-t3q3"
server_key: "677fd9664771bce405703bab0d3a497c"
server_key_iv: "81a351f81c8962b0ddb501b909c6c985"
server_key_version: "encryptionkey@0"
I20260812 01:23:17.319779 27235 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 01:23:17.320566 27260 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:23:17.320732 27235 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 01:23:17.320825 27235 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786497534195564-24578-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786497534195564-24578-0/minicluster-data/master-0/wal
uuid: "aad578a4790d4377909bdd357bee7343"
format_stamp: "Formatted at 2026-08-12 01:23:17 on dist-test-slave-t3q3"
server_key: "677fd9664771bce405703bab0d3a497c"
server_key_iv: "81a351f81c8962b0ddb501b909c6c985"
server_key_version: "encryptionkey@0"
I20260812 01:23:17.320881 27235 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786497534195564-24578-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786497534195564-24578-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786497534195564-24578-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:23:17.324172 27235 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:23:17.326180 27235 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:23:17.326287 27235 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:23:17.330180 27235 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37159
I20260812 01:23:17.330250 27348 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37159 every 8 connection(s)
I20260812 01:23:17.330566 27235 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
I20260812 01:23:17.331616 27349 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:23:17.334064 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 27235
I20260812 01:23:17.334062 27349 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aad578a4790d4377909bdd357bee7343: Bootstrap starting.
I20260812 01:23:17.334197 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786497534195564-24578-0/minicluster-data/master-0/wal/instance
I20260812 01:23:17.334379 24578 external_mini_cluster.cc:1428] Setting key 4d55f34c6d5b96ce2f5a118127106356
I20260812 01:23:17.334762 27349 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aad578a4790d4377909bdd357bee7343: Neither blocks nor log segments found. Creating new log.
I20260812 01:23:17.335361 27349 log.cc:824] T 00000000000000000000000000000000 P aad578a4790d4377909bdd357bee7343: Log is configured to *not* fsync() on all Append() calls
I20260812 01:23:17.336460 27349 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aad578a4790d4377909bdd357bee7343: No bootstrap required, opened a new log
I20260812 01:23:17.337517 27353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:17.334917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.50:40155 (local address 127.0.0.1:37159)
0812 01:23:17.335181 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:17.335189 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:17.335215 (+ 26us) server_negotiation.cc:407] Connection header received
0812 01:23:17.336141 (+ 926us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:17.336152 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:17.336192 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:17.336324 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:23:17.337052 (+ 728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:23:17.337056 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:23:17.337056 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:23:17.337059 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:23:17.337095 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:23:17.337105 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:23:17.337166 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:17.337260 (+ 94us) server_negotiation.cc:299] Negotiation successful
0812 01:23:17.337317 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":123,"threads_started":1}
I20260812 01:23:17.338521 24578 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 01:23:17.339473 27349 raft_consensus.cc:348] T 00000000000000000000000000000000 P aad578a4790d4377909bdd357bee7343 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aad578a4790d4377909bdd357bee7343" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37159 } }
I20260812 01:23:17.339664 27349 raft_consensus.cc:374] T 00000000000000000000000000000000 P aad578a4790d4377909bdd357bee7343 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:23:17.339715 27349 raft_consensus.cc:729] T 00000000000000000000000000000000 P aad578a4790d4377909bdd357bee7343 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aad578a4790d4377909bdd357bee7343, State: Initialized, Role: FOLLOWER
I20260812 01:23:17.339910 27349 consensus_queue.cc:260] T 00000000000000000000000000000000 P aad578a4790d4377909bdd357bee7343 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aad578a4790d4377909bdd357bee7343" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37159 } }
I20260812 01:23:17.340016 27349 raft_consensus.cc:388] T 00000000000000000000000000000000 P aad578a4790d4377909bdd357bee7343 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:23:17.340052 27349 raft_consensus.cc:482] T 00000000000000000000000000000000 P aad578a4790d4377909bdd357bee7343 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:23:17.340143 27349 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aad578a4790d4377909bdd357bee7343 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:23:17.340713 27349 raft_consensus.cc:504] T 00000000000000000000000000000000 P aad578a4790d4377909bdd357bee7343 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aad578a4790d4377909bdd357bee7343" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37159 } }
I20260812 01:23:17.340826 27349 leader_election.cc:302] T 00000000000000000000000000000000 P aad578a4790d4377909bdd357bee7343 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aad578a4790d4377909bdd357bee7343; no voters:
I20260812 01:23:17.340998 27349 leader_election.cc:288] T 00000000000000000000000000000000 P aad578a4790d4377909bdd357bee7343 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 01:23:17.341089 27363 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aad578a4790d4377909bdd357bee7343 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:23:17.341254 27349 sys_catalog.cc:564] T 00000000000000000000000000000000 P aad578a4790d4377909bdd357bee7343 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:23:17.341306 27363 raft_consensus.cc:686] T 00000000000000000000000000000000 P aad578a4790d4377909bdd357bee7343 [term 1 LEADER]: Becoming Leader. State: Replica: aad578a4790d4377909bdd357bee7343, State: Running, Role: LEADER
I20260812 01:23:17.341444 27353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:17.340219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.50:52663 (local address 127.0.0.1:37159)
0812 01:23:17.340348 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:17.340350 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:17.340394 (+ 44us) server_negotiation.cc:407] Connection header received
0812 01:23:17.340472 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:17.340473 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:17.340478 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:17.340522 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:23:17.341196 (+ 674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:23:17.341197 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:23:17.341198 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:23:17.341202 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:23:17.341215 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:23:17.341221 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:23:17.341253 (+ 32us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:17.341373 (+ 120us) server_negotiation.cc:299] Negotiation successful
0812 01:23:17.341387 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108}
I20260812 01:23:17.341712 27363 consensus_queue.cc:237] T 00000000000000000000000000000000 P aad578a4790d4377909bdd357bee7343 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aad578a4790d4377909bdd357bee7343" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37159 } }
I20260812 01:23:17.342329 27367 sys_catalog.cc:455] T 00000000000000000000000000000000 P aad578a4790d4377909bdd357bee7343 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aad578a4790d4377909bdd357bee7343" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aad578a4790d4377909bdd357bee7343" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37159 } } }
I20260812 01:23:17.342430 27367 sys_catalog.cc:458] T 00000000000000000000000000000000 P aad578a4790d4377909bdd357bee7343 [sys.catalog]: This master's current role is: LEADER
I20260812 01:23:17.342808 27391 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:23:17.342916 27369 sys_catalog.cc:455] T 00000000000000000000000000000000 P aad578a4790d4377909bdd357bee7343 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aad578a4790d4377909bdd357bee7343. Latest consensus state: current_term: 1 leader_uuid: "aad578a4790d4377909bdd357bee7343" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aad578a4790d4377909bdd357bee7343" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37159 } } }
I20260812 01:23:17.343161 27369 sys_catalog.cc:458] T 00000000000000000000000000000000 P aad578a4790d4377909bdd357bee7343 [sys.catalog]: This master's current role is: LEADER
I20260812 01:23:17.343367 27391 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:23:17.345026 27391 catalog_manager.cc:1347] Generated new cluster ID: 36357fabdc584453af6c8e0814656032
I20260812 01:23:17.345082 27391 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 01:23:17.353344 27391 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:23:17.353500 27391 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:23:17.357656 27391 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aad578a4790d4377909bdd357bee7343: Generated new TSK 0
I20260812 01:23:17.357821 27391 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 01:23:17.366539 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 27235
2026-08-12T01:23:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1549 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786497534195564-24578-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 01:23:17 dist-test-slave-t3q3 krb5kdc[27403](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 01:23:17 dist-test-slave-t3q3 krb5kdc[27403](info): set up 2 sockets
Aug 12 01:23:17 dist-test-slave-t3q3 krb5kdc[27403](info): commencing operation
krb5kdc: starting...
W20260812 01:23:18.631608 24578 mini_kdc.cc:121] Time spent starting KDC: real 1.238s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786497534195564-24578-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786497534195564-24578-0/krb5kdc/test-user.keytab.
Aug 12 01:23:18 dist-test-slave-t3q3 krb5kdc[27403](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497798, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T01:23:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:23:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786497534195564-24578-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786497534195564-24578-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 01:23:18.758762 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786497534195564-24578-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786497534195564-24578-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786497534195564-24578-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39967
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786497534195564-24578-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39967
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:23:18.841729 27421 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:23:18.841924 27421 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:23:18.841948 27421 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:23:18.843099 27421 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:23:18.843142 27421 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:23:18.843156 27421 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:23:18.843197 27421 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:23:18.843226 27421 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:23:18.844691 27421 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786497534195564-24578-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786497534195564-24578-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39967
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786497534195564-24578-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39967
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27421
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786497534195564-24578-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:23:18.844936 27421 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:23:18.845180 27421 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:23:18.849895 27433 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:23:18.849922 27430 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:23:18.849973 27435 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:23:18.850095 27421 server_base.cc:1034] running on GCE node
I20260812 01:23:18.850320 27421 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:23:18.850549 27421 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:23:18.851687 27421 hybrid_clock.cc:648] HybridClock initialized: now 1786497798851675 us; error 28 us; skew 500 ppm
Aug 12 01:23:18 dist-test-slave-t3q3 krb5kdc[27403](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497798, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:23:18.854226 27421 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 01:23:18.854693 27421 webserver.cc:460] Webserver started at http://127.0.0.1:44521/ using document root <none> and password file <none>
I20260812 01:23:18.854936 27421 fs_manager.cc:359] Metadata directory not provided
I20260812 01:23:18.855005 27421 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:23:18.855159 27421 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:23:18.856267 27421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786497534195564-24578-0/minicluster-data/master-0/data/instance:
uuid: "7f76637557d347c9885393c53d02f776"
format_stamp: "Formatted at 2026-08-12 01:23:18 on dist-test-slave-t3q3"
server_key: "b1726d4cb850f9e39ff0987be0528c93"
server_key_iv: "71240baf8e108813b21639f68aefe470"
server_key_version: "encryptionkey@0"
I20260812 01:23:18.856637 27421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786497534195564-24578-0/minicluster-data/master-0/wal/instance:
uuid: "7f76637557d347c9885393c53d02f776"
format_stamp: "Formatted at 2026-08-12 01:23:18 on dist-test-slave-t3q3"
server_key: "b1726d4cb850f9e39ff0987be0528c93"
server_key_iv: "71240baf8e108813b21639f68aefe470"
server_key_version: "encryptionkey@0"
I20260812 01:23:18.857918 27421 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 01:23:18.858635 27445 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:23:18.858816 27421 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 01:23:18.858907 27421 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786497534195564-24578-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786497534195564-24578-0/minicluster-data/master-0/wal
uuid: "7f76637557d347c9885393c53d02f776"
format_stamp: "Formatted at 2026-08-12 01:23:18 on dist-test-slave-t3q3"
server_key: "b1726d4cb850f9e39ff0987be0528c93"
server_key_iv: "71240baf8e108813b21639f68aefe470"
server_key_version: "encryptionkey@0"
I20260812 01:23:18.858989 27421 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786497534195564-24578-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786497534195564-24578-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786497534195564-24578-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:23:18.868749 27421 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:23:18.870882 27421 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:23:18.871033 27421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:23:18.875008 27421 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39967
I20260812 01:23:18.875082 27534 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39967 every 8 connection(s)
I20260812 01:23:18.875432 27421 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
I20260812 01:23:18.876366 27535 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:23:18.878549 27535 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f76637557d347c9885393c53d02f776: Bootstrap starting.
I20260812 01:23:18.879120 27535 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7f76637557d347c9885393c53d02f776: Neither blocks nor log segments found. Creating new log.
I20260812 01:23:18.879443 27535 log.cc:824] T 00000000000000000000000000000000 P 7f76637557d347c9885393c53d02f776: Log is configured to *not* fsync() on all Append() calls
I20260812 01:23:18.880213 27535 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f76637557d347c9885393c53d02f776: No bootstrap required, opened a new log
I20260812 01:23:18.881786 27535 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7f76637557d347c9885393c53d02f776 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f76637557d347c9885393c53d02f776" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39967 } }
I20260812 01:23:18.881915 27535 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7f76637557d347c9885393c53d02f776 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:23:18.881965 27535 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7f76637557d347c9885393c53d02f776 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f76637557d347c9885393c53d02f776, State: Initialized, Role: FOLLOWER
I20260812 01:23:18.882136 27535 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7f76637557d347c9885393c53d02f776 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f76637557d347c9885393c53d02f776" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39967 } }
I20260812 01:23:18.882200 27535 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7f76637557d347c9885393c53d02f776 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:23:18.882241 27535 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7f76637557d347c9885393c53d02f776 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:23:18.882292 27535 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7f76637557d347c9885393c53d02f776 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:23:18.882812 27535 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7f76637557d347c9885393c53d02f776 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f76637557d347c9885393c53d02f776" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39967 } }
I20260812 01:23:18.882925 27535 leader_election.cc:302] T 00000000000000000000000000000000 P 7f76637557d347c9885393c53d02f776 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7f76637557d347c9885393c53d02f776; no voters:
I20260812 01:23:18.883106 27535 leader_election.cc:288] T 00000000000000000000000000000000 P 7f76637557d347c9885393c53d02f776 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 01:23:18.883138 27540 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7f76637557d347c9885393c53d02f776 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:23:18.883370 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 27421
I20260812 01:23:18.883461 27535 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7f76637557d347c9885393c53d02f776 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:23:18.883502 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786497534195564-24578-0/minicluster-data/master-0/wal/instance
I20260812 01:23:18.883682 24578 external_mini_cluster.cc:1428] Setting key 9b584766927ad3c9b5dab251ca78a6b9
I20260812 01:23:18.883891 27540 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7f76637557d347c9885393c53d02f776 [term 1 LEADER]: Becoming Leader. State: Replica: 7f76637557d347c9885393c53d02f776, State: Running, Role: LEADER
I20260812 01:23:18.884011 27540 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7f76637557d347c9885393c53d02f776 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f76637557d347c9885393c53d02f776" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39967 } }
I20260812 01:23:18.884397 27540 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f76637557d347c9885393c53d02f776 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7f76637557d347c9885393c53d02f776. Latest consensus state: current_term: 1 leader_uuid: "7f76637557d347c9885393c53d02f776" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f76637557d347c9885393c53d02f776" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39967 } } }
I20260812 01:23:18.884509 27540 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f76637557d347c9885393c53d02f776 [sys.catalog]: This master's current role is: LEADER
I20260812 01:23:18.884639 27542 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f76637557d347c9885393c53d02f776 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7f76637557d347c9885393c53d02f776" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f76637557d347c9885393c53d02f776" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39967 } } }
I20260812 01:23:18.884790 27542 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f76637557d347c9885393c53d02f776 [sys.catalog]: This master's current role is: LEADER
W20260812 01:23:18.885409 27570 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7f76637557d347c9885393c53d02f776: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 01:23:18.885519 27570 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 01:23:18.886091 27563 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:23:18.886308 27563 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:23:18.887830 27563 catalog_manager.cc:1347] Generated new cluster ID: 6bbc4900fdf444779490021909aa8044
I20260812 01:23:18.887882 27563 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 01:23:18 dist-test-slave-t3q3 krb5kdc[27403](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497798, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 01:23:18.895947 27556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:18.884422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.50:51371 (local address 127.0.0.1:39967)
0812 01:23:18.884904 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:18.884913 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:18.884950 (+ 37us) server_negotiation.cc:407] Connection header received
0812 01:23:18.885070 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:18.885077 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:18.885138 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:18.885290 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:23:18.886163 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:18.887507 (+ 1344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:18.888601 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:18.889200 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:18.889299 (+ 99us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:18.892115 (+ 2816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:23:18.892120 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:23:18.892120 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:23:18.892134 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:23:18.894200 (+ 2066us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:23:18.894652 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:23:18.894656 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:23:18.894656 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:23:18.894784 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:23:18.895028 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:23:18.895029 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:23:18.895029 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:23:18.895348 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:23:18.895540 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:18.895624 (+ 84us) server_negotiation.cc:299] Negotiation successful
0812 01:23:18.895738 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":436,"thread_start_us":84,"threads_started":1}
I20260812 01:23:18.896986 24578 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 01:23:18.901360 27563 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:23:18.901958 27563 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:23:18.905328 27563 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7f76637557d347c9885393c53d02f776: Generated new TSK 0
I20260812 01:23:18.905501 27563 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 01:23:18.906023 27556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:18.898482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.50:38493 (local address 127.0.0.1:39967)
0812 01:23:18.898516 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:18.898518 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:18.898614 (+ 96us) server_negotiation.cc:407] Connection header received
0812 01:23:18.898660 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:18.898661 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:18.898671 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:18.898728 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:23:18.899304 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:18.900136 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:18.901081 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:18.901414 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:18.901481 (+ 67us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:18.902995 (+ 1514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:23:18.902996 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:23:18.902997 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:23:18.903003 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:23:18.904592 (+ 1589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:23:18.905044 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:23:18.905045 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:23:18.905046 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:23:18.905102 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:23:18.905390 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:23:18.905391 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:23:18.905391 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:23:18.905516 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:23:18.905618 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:18.905857 (+ 239us) server_negotiation.cc:299] Negotiation successful
0812 01:23:18.905923 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 01:23:18.908293 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 27421
W20260812 01:23:18.932086 27401 connection.cc:535] client connection to 127.0.0.1:39967 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39967): Connection reset by peer (error 104)
2026-08-12T01:23:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1552 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786497534195564-24578-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 01:23:18 dist-test-slave-t3q3 krb5kdc[27584](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 01:23:18 dist-test-slave-t3q3 krb5kdc[27584](info): set up 2 sockets
Aug 12 01:23:18 dist-test-slave-t3q3 krb5kdc[27584](info): commencing operation
krb5kdc: starting...
W20260812 01:23:20.235380 24578 mini_kdc.cc:121] Time spent starting KDC: real 1.289s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786497534195564-24578-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786497534195564-24578-0/krb5kdc/test-user.keytab.
Aug 12 01:23:20 dist-test-slave-t3q3 krb5kdc[27584](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497800, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T01:23:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:23:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786497534195564-24578-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786497534195564-24578-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 01:23:20.360683 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786497534195564-24578-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786497534195564-24578-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786497534195564-24578-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33865
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786497534195564-24578-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33865
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:23:20.445293 27607 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:23:20.445541 27607 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:23:20.445580 27607 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:23:20.446736 27607 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:23:20.446807 27607 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:23:20.446831 27607 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:23:20.446852 27607 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:23:20.446885 27607 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:23:20.448428 27607 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786497534195564-24578-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786497534195564-24578-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33865
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786497534195564-24578-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33865
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27607
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786497534195564-24578-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:23:20.448714 27607 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:23:20.448953 27607 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:23:20.453980 27617 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:23:20.454005 27620 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:23:20.453989 27615 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:23:20.454190 27607 server_base.cc:1034] running on GCE node
I20260812 01:23:20.454502 27607 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:23:20.454800 27607 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:23:20.455965 27607 hybrid_clock.cc:648] HybridClock initialized: now 1786497800455944 us; error 52 us; skew 500 ppm
Aug 12 01:23:20 dist-test-slave-t3q3 krb5kdc[27584](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497800, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:23:20.458602 27607 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 01:23:20.459069 27607 webserver.cc:460] Webserver started at http://127.0.0.1:38607/ using document root <none> and password file <none>
I20260812 01:23:20.459277 27607 fs_manager.cc:359] Metadata directory not provided
I20260812 01:23:20.459319 27607 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:23:20.459412 27607 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:23:20.460534 27607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786497534195564-24578-0/minicluster-data/master-0/data/instance:
uuid: "2e810b411bf84ebbb8752c4aa2ebfec7"
format_stamp: "Formatted at 2026-08-12 01:23:20 on dist-test-slave-t3q3"
server_key: "bf92c981eca6b11e418f8d99f3319814"
server_key_iv: "2ee978949fa5e6e2dea17239f0febec9"
server_key_version: "encryptionkey@0"
I20260812 01:23:20.460856 27607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786497534195564-24578-0/minicluster-data/master-0/wal/instance:
uuid: "2e810b411bf84ebbb8752c4aa2ebfec7"
format_stamp: "Formatted at 2026-08-12 01:23:20 on dist-test-slave-t3q3"
server_key: "bf92c981eca6b11e418f8d99f3319814"
server_key_iv: "2ee978949fa5e6e2dea17239f0febec9"
server_key_version: "encryptionkey@0"
I20260812 01:23:20.462178 27607 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 01:23:20.462889 27630 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:23:20.463088 27607 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:23:20.463178 27607 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786497534195564-24578-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786497534195564-24578-0/minicluster-data/master-0/wal
uuid: "2e810b411bf84ebbb8752c4aa2ebfec7"
format_stamp: "Formatted at 2026-08-12 01:23:20 on dist-test-slave-t3q3"
server_key: "bf92c981eca6b11e418f8d99f3319814"
server_key_iv: "2ee978949fa5e6e2dea17239f0febec9"
server_key_version: "encryptionkey@0"
I20260812 01:23:20.463238 27607 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786497534195564-24578-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786497534195564-24578-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786497534195564-24578-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:23:20.488808 27607 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:23:20.490949 27607 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:23:20.491071 27607 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:23:20.495157 27607 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33865
I20260812 01:23:20.495223 27717 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33865 every 8 connection(s)
I20260812 01:23:20.495590 27607 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
I20260812 01:23:20.495844 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 27607
I20260812 01:23:20.495977 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786497534195564-24578-0/minicluster-data/master-0/wal/instance
I20260812 01:23:20.496297 24578 external_mini_cluster.cc:1428] Setting key 95b8e3abc68c9b346ba5a7b3d91bb23e
I20260812 01:23:20.496718 27718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:23:20.499903 27718 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e810b411bf84ebbb8752c4aa2ebfec7: Bootstrap starting.
I20260812 01:23:20.500643 27718 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2e810b411bf84ebbb8752c4aa2ebfec7: Neither blocks nor log segments found. Creating new log.
I20260812 01:23:20.501019 27718 log.cc:824] T 00000000000000000000000000000000 P 2e810b411bf84ebbb8752c4aa2ebfec7: Log is configured to *not* fsync() on all Append() calls
I20260812 01:23:20.501744 27721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:20.496822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.50:44173 (local address 127.0.0.1:33865)
0812 01:23:20.497020 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:20.497028 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:20.497053 (+ 25us) server_negotiation.cc:407] Connection header received
0812 01:23:20.497475 (+ 422us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:20.497481 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:20.497526 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:20.497645 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:23:20.497869 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:20.498740 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:20.499842 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:20.500328 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:20.500418 (+ 90us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:20.501083 (+ 665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:23:20.501087 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:23:20.501087 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:23:20.501290 (+ 203us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:23:20.501322 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:23:20.501332 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:23:20.501430 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:20.501518 (+ 88us) server_negotiation.cc:299] Negotiation successful
0812 01:23:20.501545 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":101,"threads_started":1}
I20260812 01:23:20.501818 27718 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e810b411bf84ebbb8752c4aa2ebfec7: No bootstrap required, opened a new log
I20260812 01:23:20.502743 24578 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 01:23:20.504130 27718 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2e810b411bf84ebbb8752c4aa2ebfec7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e810b411bf84ebbb8752c4aa2ebfec7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33865 } }
I20260812 01:23:20.504304 27718 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2e810b411bf84ebbb8752c4aa2ebfec7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:23:20.504359 27718 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2e810b411bf84ebbb8752c4aa2ebfec7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e810b411bf84ebbb8752c4aa2ebfec7, State: Initialized, Role: FOLLOWER
I20260812 01:23:20.504482 27718 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2e810b411bf84ebbb8752c4aa2ebfec7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e810b411bf84ebbb8752c4aa2ebfec7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33865 } }
I20260812 01:23:20.504561 27718 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2e810b411bf84ebbb8752c4aa2ebfec7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:23:20.504593 27718 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2e810b411bf84ebbb8752c4aa2ebfec7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:23:20.504648 27718 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2e810b411bf84ebbb8752c4aa2ebfec7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:23:20.505322 27718 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2e810b411bf84ebbb8752c4aa2ebfec7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e810b411bf84ebbb8752c4aa2ebfec7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33865 } }
I20260812 01:23:20.505441 27718 leader_election.cc:302] T 00000000000000000000000000000000 P 2e810b411bf84ebbb8752c4aa2ebfec7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2e810b411bf84ebbb8752c4aa2ebfec7; no voters:
I20260812 01:23:20.505615 27718 leader_election.cc:288] T 00000000000000000000000000000000 P 2e810b411bf84ebbb8752c4aa2ebfec7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 01:23:20.505671 27732 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2e810b411bf84ebbb8752c4aa2ebfec7 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:23:20.505836 27732 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2e810b411bf84ebbb8752c4aa2ebfec7 [term 1 LEADER]: Becoming Leader. State: Replica: 2e810b411bf84ebbb8752c4aa2ebfec7, State: Running, Role: LEADER
I20260812 01:23:20.505897 27718 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2e810b411bf84ebbb8752c4aa2ebfec7 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:23:20.505995 27732 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2e810b411bf84ebbb8752c4aa2ebfec7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e810b411bf84ebbb8752c4aa2ebfec7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33865 } }
I20260812 01:23:20.506381 27732 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e810b411bf84ebbb8752c4aa2ebfec7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2e810b411bf84ebbb8752c4aa2ebfec7. Latest consensus state: current_term: 1 leader_uuid: "2e810b411bf84ebbb8752c4aa2ebfec7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e810b411bf84ebbb8752c4aa2ebfec7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33865 } } }
I20260812 01:23:20.506528 27732 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e810b411bf84ebbb8752c4aa2ebfec7 [sys.catalog]: This master's current role is: LEADER
I20260812 01:23:20.507676 27721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:20.504052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.50:55869 (local address 127.0.0.1:33865)
0812 01:23:20.504110 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:20.504112 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:20.504248 (+ 136us) server_negotiation.cc:407] Connection header received
0812 01:23:20.504323 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:20.504324 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:20.504336 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:20.504385 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:23:20.504578 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:20.505321 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:20.506275 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:20.506562 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:20.506633 (+ 71us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:20.507345 (+ 712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:23:20.507346 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:23:20.507347 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:23:20.507352 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:23:20.507366 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:23:20.507371 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:23:20.507422 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:20.507593 (+ 171us) server_negotiation.cc:299] Negotiation successful
0812 01:23:20.507601 (+ 8us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":9,"server-negotiator.queue_time_us":41}
I20260812 01:23:20.507701 27752 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:23:20.507781 27733 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e810b411bf84ebbb8752c4aa2ebfec7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2e810b411bf84ebbb8752c4aa2ebfec7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e810b411bf84ebbb8752c4aa2ebfec7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33865 } } }
I20260812 01:23:20.507869 27733 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e810b411bf84ebbb8752c4aa2ebfec7 [sys.catalog]: This master's current role is: LEADER
I20260812 01:23:20.508265 27752 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:23:20.509850 27752 catalog_manager.cc:1347] Generated new cluster ID: 2c71b50d0eba49bd9a6ccbcea98f6df3
I20260812 01:23:20.509903 27752 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 01:23:20.515035 27752 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:23:20.515652 27752 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:23:20.519069 27752 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2e810b411bf84ebbb8752c4aa2ebfec7: Generated new TSK 0
I20260812 01:23:20.519239 27752 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 01:23:20.521636 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 27607
W20260812 01:23:20.537974 27581 connection.cc:535] client connection to 127.0.0.1:33865 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33865): Connection reset by peer (error 104)
2026-08-12T01:23:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1605 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786497534195564-24578-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 01:23:20 dist-test-slave-t3q3 krb5kdc[27780](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 01:23:20 dist-test-slave-t3q3 krb5kdc[27780](info): set up 2 sockets
Aug 12 01:23:20 dist-test-slave-t3q3 krb5kdc[27780](info): commencing operation
krb5kdc: starting...
W20260812 01:23:21.788697 24578 mini_kdc.cc:121] Time spent starting KDC: real 1.237s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786497534195564-24578-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786497534195564-24578-0/krb5kdc/test-user.keytab.
Aug 12 01:23:21 dist-test-slave-t3q3 krb5kdc[27780](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497801, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T01:23:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:23:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786497534195564-24578-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786497534195564-24578-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 01:23:21.910426 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786497534195564-24578-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786497534195564-24578-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786497534195564-24578-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43383
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786497534195564-24578-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43383
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:23:21.994323 27804 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:23:21.994561 27804 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:23:21.994587 27804 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:23:21.995838 27804 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:23:21.995883 27804 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:23:21.995899 27804 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:23:21.995913 27804 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:23:21.995924 27804 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:23:21.997426 27804 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786497534195564-24578-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786497534195564-24578-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43383
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786497534195564-24578-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43383
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27804
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786497534195564-24578-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:23:21.997694 27804 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:23:21.997928 27804 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:23:22.003149 27811 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:23:22.003122 27816 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:23:22.003208 27804 server_base.cc:1034] running on GCE node
W20260812 01:23:22.003146 27810 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:23:22.003628 27804 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:23:22.003928 27804 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:23:22.005091 27804 hybrid_clock.cc:648] HybridClock initialized: now 1786497802005066 us; error 36 us; skew 500 ppm
Aug 12 01:23:22 dist-test-slave-t3q3 krb5kdc[27780](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497802, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:23:22.011835 27804 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 01:23:22.012348 27804 webserver.cc:460] Webserver started at http://127.0.0.1:35315/ using document root <none> and password file <none>
I20260812 01:23:22.012578 27804 fs_manager.cc:359] Metadata directory not provided
I20260812 01:23:22.012624 27804 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:23:22.012786 27804 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:23:22.013881 27804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786497534195564-24578-0/minicluster-data/master-0/data/instance:
uuid: "50e1ed4321e3432d9484aadafc6b866a"
format_stamp: "Formatted at 2026-08-12 01:23:22 on dist-test-slave-t3q3"
server_key: "3ba6a4b857b329fcec8f1e45a48dec53"
server_key_iv: "f42679377d4bcbb592e1fe59acf79ca1"
server_key_version: "encryptionkey@0"
I20260812 01:23:22.014259 27804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786497534195564-24578-0/minicluster-data/master-0/wal/instance:
uuid: "50e1ed4321e3432d9484aadafc6b866a"
format_stamp: "Formatted at 2026-08-12 01:23:22 on dist-test-slave-t3q3"
server_key: "3ba6a4b857b329fcec8f1e45a48dec53"
server_key_iv: "f42679377d4bcbb592e1fe59acf79ca1"
server_key_version: "encryptionkey@0"
I20260812 01:23:22.015637 27804 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 01:23:22.016466 27828 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:23:22.016635 27804 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 01:23:22.016743 27804 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786497534195564-24578-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786497534195564-24578-0/minicluster-data/master-0/wal
uuid: "50e1ed4321e3432d9484aadafc6b866a"
format_stamp: "Formatted at 2026-08-12 01:23:22 on dist-test-slave-t3q3"
server_key: "3ba6a4b857b329fcec8f1e45a48dec53"
server_key_iv: "f42679377d4bcbb592e1fe59acf79ca1"
server_key_version: "encryptionkey@0"
I20260812 01:23:22.016829 27804 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786497534195564-24578-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786497534195564-24578-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786497534195564-24578-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:23:22.020128 27804 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:23:22.022114 27804 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:23:22.022282 27804 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:23:22.026432 27804 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43383
I20260812 01:23:22.026464 27919 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43383 every 8 connection(s)
I20260812 01:23:22.026794 27804 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
I20260812 01:23:22.027761 27921 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:23:22.030055 27921 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50e1ed4321e3432d9484aadafc6b866a: Bootstrap starting.
I20260812 01:23:22.030639 27921 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 50e1ed4321e3432d9484aadafc6b866a: Neither blocks nor log segments found. Creating new log.
I20260812 01:23:22.030936 27921 log.cc:824] T 00000000000000000000000000000000 P 50e1ed4321e3432d9484aadafc6b866a: Log is configured to *not* fsync() on all Append() calls
I20260812 01:23:22.031620 27921 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50e1ed4321e3432d9484aadafc6b866a: No bootstrap required, opened a new log
I20260812 01:23:22.033286 27921 raft_consensus.cc:348] T 00000000000000000000000000000000 P 50e1ed4321e3432d9484aadafc6b866a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50e1ed4321e3432d9484aadafc6b866a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43383 } }
I20260812 01:23:22.033420 27921 raft_consensus.cc:374] T 00000000000000000000000000000000 P 50e1ed4321e3432d9484aadafc6b866a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:23:22.033444 27921 raft_consensus.cc:729] T 00000000000000000000000000000000 P 50e1ed4321e3432d9484aadafc6b866a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50e1ed4321e3432d9484aadafc6b866a, State: Initialized, Role: FOLLOWER
I20260812 01:23:22.033620 27921 consensus_queue.cc:260] T 00000000000000000000000000000000 P 50e1ed4321e3432d9484aadafc6b866a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50e1ed4321e3432d9484aadafc6b866a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43383 } }
I20260812 01:23:22.033692 27921 raft_consensus.cc:388] T 00000000000000000000000000000000 P 50e1ed4321e3432d9484aadafc6b866a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:23:22.033733 27921 raft_consensus.cc:482] T 00000000000000000000000000000000 P 50e1ed4321e3432d9484aadafc6b866a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:23:22.033783 27921 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 50e1ed4321e3432d9484aadafc6b866a [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:23:22.034327 27921 raft_consensus.cc:504] T 00000000000000000000000000000000 P 50e1ed4321e3432d9484aadafc6b866a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50e1ed4321e3432d9484aadafc6b866a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43383 } }
I20260812 01:23:22.034440 27921 leader_election.cc:302] T 00000000000000000000000000000000 P 50e1ed4321e3432d9484aadafc6b866a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 50e1ed4321e3432d9484aadafc6b866a; no voters:
I20260812 01:23:22.034641 27921 leader_election.cc:288] T 00000000000000000000000000000000 P 50e1ed4321e3432d9484aadafc6b866a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 01:23:22.034703 27929 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 50e1ed4321e3432d9484aadafc6b866a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:23:22.034881 27929 raft_consensus.cc:686] T 00000000000000000000000000000000 P 50e1ed4321e3432d9484aadafc6b866a [term 1 LEADER]: Becoming Leader. State: Replica: 50e1ed4321e3432d9484aadafc6b866a, State: Running, Role: LEADER
I20260812 01:23:22.034904 27921 sys_catalog.cc:564] T 00000000000000000000000000000000 P 50e1ed4321e3432d9484aadafc6b866a [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:23:22.035018 27929 consensus_queue.cc:237] T 00000000000000000000000000000000 P 50e1ed4321e3432d9484aadafc6b866a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50e1ed4321e3432d9484aadafc6b866a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43383 } }
I20260812 01:23:22.035377 27930 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50e1ed4321e3432d9484aadafc6b866a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "50e1ed4321e3432d9484aadafc6b866a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50e1ed4321e3432d9484aadafc6b866a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43383 } } }
I20260812 01:23:22.035480 27930 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50e1ed4321e3432d9484aadafc6b866a [sys.catalog]: This master's current role is: LEADER
I20260812 01:23:22.035386 27931 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50e1ed4321e3432d9484aadafc6b866a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 50e1ed4321e3432d9484aadafc6b866a. Latest consensus state: current_term: 1 leader_uuid: "50e1ed4321e3432d9484aadafc6b866a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50e1ed4321e3432d9484aadafc6b866a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43383 } } }
I20260812 01:23:22.035567 27931 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50e1ed4321e3432d9484aadafc6b866a [sys.catalog]: This master's current role is: LEADER
I20260812 01:23:22.035928 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 27804
I20260812 01:23:22.035732 27934 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:23:22.036043 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786497534195564-24578-0/minicluster-data/master-0/wal/instance
I20260812 01:23:22.036249 24578 external_mini_cluster.cc:1428] Setting key 118c8e927d9903d6c6a5346f8ea7c679
I20260812 01:23:22.036526 27934 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:23:22.038348 27934 catalog_manager.cc:1347] Generated new cluster ID: 3de3211d7b2f438ab01a95bc199ca0e0
I20260812 01:23:22.038409 27934 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 01:23:22.038822 27962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:22.036908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.50:40847 (local address 127.0.0.1:43383)
0812 01:23:22.037089 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:22.037098 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:22.037338 (+ 240us) server_negotiation.cc:407] Connection header received
0812 01:23:22.037502 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:22.037508 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:22.037529 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:22.037661 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:23:22.038351 (+ 690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:23:22.038355 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:23:22.038355 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:23:22.038360 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:23:22.038403 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:23:22.038418 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:23:22.038458 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:22.038588 (+ 130us) server_negotiation.cc:299] Negotiation successful
0812 01:23:22.038643 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":80,"threads_started":1}
I20260812 01:23:22.039752 24578 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 01:23:22.042404 27962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:22.041195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.50:39879 (local address 127.0.0.1:43383)
0812 01:23:22.041236 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:22.041238 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:22.041321 (+ 83us) server_negotiation.cc:407] Connection header received
0812 01:23:22.041426 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:22.041427 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:22.041458 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:22.041490 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:23:22.042156 (+ 666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:23:22.042157 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:23:22.042158 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:23:22.042162 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:23:22.042175 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:23:22.042180 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:23:22.042210 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:22.042324 (+ 114us) server_negotiation.cc:299] Negotiation successful
0812 01:23:22.042337 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 01:23:22.056905 27934 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:23:22.057090 27934 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:23:22.069583 27934 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 50e1ed4321e3432d9484aadafc6b866a: Generated new TSK 0
I20260812 01:23:22.069792 27934 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 01:23:22.080472 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 27804
2026-08-12T01:23:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1556 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786497534195564-24578-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 01:23:22 dist-test-slave-t3q3 krb5kdc[27972](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 01:23:22 dist-test-slave-t3q3 krb5kdc[27972](info): set up 2 sockets
Aug 12 01:23:22 dist-test-slave-t3q3 krb5kdc[27972](info): commencing operation
krb5kdc: starting...
W20260812 01:23:23.397215 24578 mini_kdc.cc:121] Time spent starting KDC: real 1.289s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786497534195564-24578-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786497534195564-24578-0/krb5kdc/test-user.keytab.
Aug 12 01:23:23 dist-test-slave-t3q3 krb5kdc[27972](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497803, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T01:23:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:23:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.0.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.0.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.0.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab.
Entry for principal kudu/127.24.0.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab.
I20260812 01:23:23.522535 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786497534195564-24578-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786497534195564-24578-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786497534195564-24578-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.190:42889
--webserver_interface=127.24.0.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab
--principal=kudu/127.24.0.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:43737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.0.190:42889
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:23:23.606946 27993 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:23:23.607266 27993 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:23:23.607298 27993 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:23:23.608490 27993 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:23:23.608533 27993 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:23:23.608547 27993 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:23:23.608560 27993 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:23:23.608572 27993 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:23:23.610080 27993 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:43737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786497534195564-24578-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786497534195564-24578-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.0.190:42889
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab
--principal=kudu/127.24.0.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.190:42889
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.0.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27993
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786497534195564-24578-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:23:23.610343 27993 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:23:23.610597 27993 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:23:23.615554 28006 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:23:23.615562 28004 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:23:23.615620 28008 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:23:23.615795 27993 server_base.cc:1034] running on GCE node
I20260812 01:23:23.616047 27993 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:23:23.616376 27993 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:23:23.617535 27993 hybrid_clock.cc:648] HybridClock initialized: now 1786497803617528 us; error 36 us; skew 500 ppm
Aug 12 01:23:23 dist-test-slave-t3q3 krb5kdc[27972](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497803, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:23:23.619995 27993 init.cc:377] Logged in from keytab as kudu/127.24.0.190@KRBTEST.COM (short username kudu)
I20260812 01:23:23.620445 27993 webserver.cc:460] Webserver started at http://127.24.0.190:45895/ using document root <none> and password file <none>
I20260812 01:23:23.620675 27993 fs_manager.cc:359] Metadata directory not provided
I20260812 01:23:23.620740 27993 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:23:23.620890 27993 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:23:23.621948 27993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786497534195564-24578-0/minicluster-data/master-0/data/instance:
uuid: "6197f8dbfcad4f4fb66a412826335d8f"
format_stamp: "Formatted at 2026-08-12 01:23:23 on dist-test-slave-t3q3"
server_key: "4b1481dfc2bcebd8c53687453cb0e5e3"
server_key_iv: "c4826fe2baf1504056ad75e4233b3880"
server_key_version: "encryptionkey@0"
I20260812 01:23:23.622323 27993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786497534195564-24578-0/minicluster-data/master-0/wal/instance:
uuid: "6197f8dbfcad4f4fb66a412826335d8f"
format_stamp: "Formatted at 2026-08-12 01:23:23 on dist-test-slave-t3q3"
server_key: "4b1481dfc2bcebd8c53687453cb0e5e3"
server_key_iv: "c4826fe2baf1504056ad75e4233b3880"
server_key_version: "encryptionkey@0"
I20260812 01:23:23.623675 27993 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 01:23:23.624447 28024 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:23:23.624612 27993 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 01:23:23.624722 27993 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786497534195564-24578-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786497534195564-24578-0/minicluster-data/master-0/wal
uuid: "6197f8dbfcad4f4fb66a412826335d8f"
format_stamp: "Formatted at 2026-08-12 01:23:23 on dist-test-slave-t3q3"
server_key: "4b1481dfc2bcebd8c53687453cb0e5e3"
server_key_iv: "c4826fe2baf1504056ad75e4233b3880"
server_key_version: "encryptionkey@0"
I20260812 01:23:23.624796 27993 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786497534195564-24578-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786497534195564-24578-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786497534195564-24578-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:23:23.637143 27993 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:23:23.638988 27993 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:23:23.639150 27993 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:23:23.643239 27993 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.190:42889
I20260812 01:23:23.643303 28110 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.190:42889 every 8 connection(s)
I20260812 01:23:23.643654 27993 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
I20260812 01:23:23.644547 28111 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:23:23.646847 28111 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6197f8dbfcad4f4fb66a412826335d8f: Bootstrap starting.
I20260812 01:23:23.647440 28111 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6197f8dbfcad4f4fb66a412826335d8f: Neither blocks nor log segments found. Creating new log.
I20260812 01:23:23.647747 28111 log.cc:824] T 00000000000000000000000000000000 P 6197f8dbfcad4f4fb66a412826335d8f: Log is configured to *not* fsync() on all Append() calls
I20260812 01:23:23.647859 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 27993
I20260812 01:23:23.647976 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786497534195564-24578-0/minicluster-data/master-0/wal/instance
I20260812 01:23:23.648198 24578 external_mini_cluster.cc:1428] Setting key 613eabf5e896c1f2ef1cad6f169acfc9
I20260812 01:23:23.648561 28111 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6197f8dbfcad4f4fb66a412826335d8f: No bootstrap required, opened a new log
I20260812 01:23:23.650323 28111 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6197f8dbfcad4f4fb66a412826335d8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6197f8dbfcad4f4fb66a412826335d8f" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 42889 } }
I20260812 01:23:23.650486 28111 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6197f8dbfcad4f4fb66a412826335d8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:23:23.650527 28111 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6197f8dbfcad4f4fb66a412826335d8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6197f8dbfcad4f4fb66a412826335d8f, State: Initialized, Role: FOLLOWER
I20260812 01:23:23.650723 28111 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6197f8dbfcad4f4fb66a412826335d8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6197f8dbfcad4f4fb66a412826335d8f" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 42889 } }
I20260812 01:23:23.650822 28111 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6197f8dbfcad4f4fb66a412826335d8f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:23:23.650861 28111 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6197f8dbfcad4f4fb66a412826335d8f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:23:23.650911 28111 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6197f8dbfcad4f4fb66a412826335d8f [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:23:23.651968 28111 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6197f8dbfcad4f4fb66a412826335d8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6197f8dbfcad4f4fb66a412826335d8f" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 42889 } }
I20260812 01:23:23.652109 28111 leader_election.cc:302] T 00000000000000000000000000000000 P 6197f8dbfcad4f4fb66a412826335d8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6197f8dbfcad4f4fb66a412826335d8f; no voters:
I20260812 01:23:23.652311 28111 leader_election.cc:288] T 00000000000000000000000000000000 P 6197f8dbfcad4f4fb66a412826335d8f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 01:23:23.652335 28118 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6197f8dbfcad4f4fb66a412826335d8f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:23:23.652449 28118 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6197f8dbfcad4f4fb66a412826335d8f [term 1 LEADER]: Becoming Leader. State: Replica: 6197f8dbfcad4f4fb66a412826335d8f, State: Running, Role: LEADER
I20260812 01:23:23.652582 28111 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6197f8dbfcad4f4fb66a412826335d8f [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:23:23.652616 28118 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6197f8dbfcad4f4fb66a412826335d8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6197f8dbfcad4f4fb66a412826335d8f" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 42889 } }
I20260812 01:23:23.653052 28122 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6197f8dbfcad4f4fb66a412826335d8f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6197f8dbfcad4f4fb66a412826335d8f. Latest consensus state: current_term: 1 leader_uuid: "6197f8dbfcad4f4fb66a412826335d8f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6197f8dbfcad4f4fb66a412826335d8f" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 42889 } } }
I20260812 01:23:23.653146 28122 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6197f8dbfcad4f4fb66a412826335d8f [sys.catalog]: This master's current role is: LEADER
I20260812 01:23:23.653594 28121 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6197f8dbfcad4f4fb66a412826335d8f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6197f8dbfcad4f4fb66a412826335d8f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6197f8dbfcad4f4fb66a412826335d8f" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 42889 } } }
I20260812 01:23:23.653679 28121 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6197f8dbfcad4f4fb66a412826335d8f [sys.catalog]: This master's current role is: LEADER
I20260812 01:23:23.653857 28134 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:23:23.654434 28134 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:23:23.656136 28134 catalog_manager.cc:1347] Generated new cluster ID: 6189bce6b30a4f6ea4f5a07e9c409fa4
I20260812 01:23:23.656191 28134 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 01:23:23 dist-test-slave-t3q3 krb5kdc[27972](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497803, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
I20260812 01:23:23.661671 28117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:23.648714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40789 (local address 127.24.0.190:42889)
0812 01:23:23.648896 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:23.648905 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:23.648939 (+ 34us) server_negotiation.cc:407] Connection header received
0812 01:23:23.650310 (+ 1371us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:23.650316 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:23.650363 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:23.650475 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:23:23.651352 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:23.652427 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:23.653533 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:23.654244 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:23.654388 (+ 144us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:23.657200 (+ 2812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:23:23.657204 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:23:23.657204 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:23:23.657216 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:23:23.659289 (+ 2073us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:23:23.659765 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:23:23.659768 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:23:23.659768 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:23:23.659897 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:23:23.660195 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:23:23.660196 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:23:23.660197 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:23:23.660502 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:23:23.661084 (+ 582us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:23.661184 (+ 100us) server_negotiation.cc:299] Negotiation successful
0812 01:23:23.661288 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":94,"threads_started":1}
I20260812 01:23:23.662684 24578 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 01:23:23.666970 28134 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:23:23.667639 28134 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:23:23.670953 28117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:23.664204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39501 (local address 127.24.0.190:42889)
0812 01:23:23.664239 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:23.664240 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:23.664349 (+ 109us) server_negotiation.cc:407] Connection header received
0812 01:23:23.664378 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:23.664379 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:23.664389 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:23.664457 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:23:23.665019 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:23.665763 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:23.666705 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:23.666986 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:23.667069 (+ 83us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:23.668400 (+ 1331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:23:23.668402 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:23:23.668402 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:23:23.668409 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:23:23.669791 (+ 1382us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:23:23.670147 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:23:23.670148 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:23:23.670149 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:23:23.670218 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:23:23.670437 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:23:23.670438 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:23:23.670438 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:23:23.670546 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:23:23.670669 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:23.670781 (+ 112us) server_negotiation.cc:299] Negotiation successful
0812 01:23:23.670846 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260812 01:23:23.678027 28134 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6197f8dbfcad4f4fb66a412826335d8f: Generated new TSK 0
I20260812 01:23:23.678195 28134 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 01:23:23.687083 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 27993
W20260812 01:23:23.700417 27969 connection.cc:535] client connection to 127.24.0.190:42889 recv error: Network error: failed to read from TLS socket (remote: 127.24.0.190:42889): Connection reset by peer (error 104)
2026-08-12T01:23:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1605 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786497534195564-24578-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 01:23:23 dist-test-slave-t3q3 krb5kdc[28167](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 01:23:23 dist-test-slave-t3q3 krb5kdc[28167](info): set up 2 sockets
Aug 12 01:23:23 dist-test-slave-t3q3 krb5kdc[28167](info): commencing operation
krb5kdc: starting...
W20260812 01:23:24.986996 24578 mini_kdc.cc:121] Time spent starting KDC: real 1.273s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786497534195564-24578-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786497534195564-24578-0/krb5kdc/test-user.keytab.
Aug 12 01:23:25 dist-test-slave-t3q3 krb5kdc[28167](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497805, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T01:23:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:23:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.0.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.0.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.0.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab.
Entry for principal kudu/127.24.0.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab.
I20260812 01:23:25.109035 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786497534195564-24578-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786497534195564-24578-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786497534195564-24578-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.190:34437
--webserver_interface=127.24.0.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab
--principal=kudu/127.24.0.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:42151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.0.190:34437
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:23:25.191560 28188 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:23:25.191762 28188 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:23:25.191784 28188 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:23:25.192929 28188 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:23:25.192973 28188 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:23:25.193002 28188 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:23:25.193048 28188 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:23:25.193079 28188 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:23:25.194541 28188 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:42151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786497534195564-24578-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786497534195564-24578-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.0.190:34437
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab
--principal=kudu/127.24.0.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.190:34437
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.0.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28188
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786497534195564-24578-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:23:25.194784 28188 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:23:25.195008 28188 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:23:25.199658 28195 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:23:25.199798 28188 server_base.cc:1034] running on GCE node
W20260812 01:23:25.199676 28199 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:23:25.199671 28196 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:23:25.200184 28188 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:23:25.200466 28188 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:23:25.201606 28188 hybrid_clock.cc:648] HybridClock initialized: now 1786497805201595 us; error 33 us; skew 500 ppm
Aug 12 01:23:25 dist-test-slave-t3q3 krb5kdc[28167](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497805, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:23:25.204030 28188 init.cc:377] Logged in from keytab as kudu/127.24.0.190@KRBTEST.COM (short username kudu)
I20260812 01:23:25.204492 28188 webserver.cc:460] Webserver started at http://127.24.0.190:43439/ using document root <none> and password file <none>
I20260812 01:23:25.204715 28188 fs_manager.cc:359] Metadata directory not provided
I20260812 01:23:25.204761 28188 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:23:25.204908 28188 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:23:25.205946 28188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786497534195564-24578-0/minicluster-data/master-0/data/instance:
uuid: "0444894f7616434abc980ceea53a8542"
format_stamp: "Formatted at 2026-08-12 01:23:25 on dist-test-slave-t3q3"
server_key: "48caefdead761f90ee84781bb0990cc1"
server_key_iv: "b59d074d52e3326366b4dd8414e2a7a8"
server_key_version: "encryptionkey@0"
I20260812 01:23:25.206301 28188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786497534195564-24578-0/minicluster-data/master-0/wal/instance:
uuid: "0444894f7616434abc980ceea53a8542"
format_stamp: "Formatted at 2026-08-12 01:23:25 on dist-test-slave-t3q3"
server_key: "48caefdead761f90ee84781bb0990cc1"
server_key_iv: "b59d074d52e3326366b4dd8414e2a7a8"
server_key_version: "encryptionkey@0"
I20260812 01:23:25.207649 28188 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 01:23:25.208462 28210 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:23:25.208649 28188 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 01:23:25.208748 28188 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786497534195564-24578-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786497534195564-24578-0/minicluster-data/master-0/wal
uuid: "0444894f7616434abc980ceea53a8542"
format_stamp: "Formatted at 2026-08-12 01:23:25 on dist-test-slave-t3q3"
server_key: "48caefdead761f90ee84781bb0990cc1"
server_key_iv: "b59d074d52e3326366b4dd8414e2a7a8"
server_key_version: "encryptionkey@0"
I20260812 01:23:25.208822 28188 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786497534195564-24578-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786497534195564-24578-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786497534195564-24578-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:23:25.232496 28188 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:23:25.234634 28188 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:23:25.234792 28188 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:23:25.238905 28188 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.190:34437
I20260812 01:23:25.238998 28289 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.190:34437 every 8 connection(s)
I20260812 01:23:25.239338 28188 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
I20260812 01:23:25.240298 28291 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:23:25.242655 28291 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0444894f7616434abc980ceea53a8542: Bootstrap starting.
I20260812 01:23:25.243257 28291 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0444894f7616434abc980ceea53a8542: Neither blocks nor log segments found. Creating new log.
I20260812 01:23:25.243558 28291 log.cc:824] T 00000000000000000000000000000000 P 0444894f7616434abc980ceea53a8542: Log is configured to *not* fsync() on all Append() calls
I20260812 01:23:25.244313 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 28188
I20260812 01:23:25.244346 28291 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0444894f7616434abc980ceea53a8542: No bootstrap required, opened a new log
I20260812 01:23:25.244438 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786497534195564-24578-0/minicluster-data/master-0/wal/instance
I20260812 01:23:25.244647 24578 external_mini_cluster.cc:1428] Setting key 62e0c5f4875c35bac4ae52319ab326eb
I20260812 01:23:25.246332 28291 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0444894f7616434abc980ceea53a8542 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0444894f7616434abc980ceea53a8542" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 34437 } }
I20260812 01:23:25.246506 28291 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0444894f7616434abc980ceea53a8542 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:23:25.246548 28291 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0444894f7616434abc980ceea53a8542 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0444894f7616434abc980ceea53a8542, State: Initialized, Role: FOLLOWER
I20260812 01:23:25.246755 28291 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0444894f7616434abc980ceea53a8542 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0444894f7616434abc980ceea53a8542" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 34437 } }
I20260812 01:23:25.246860 28291 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0444894f7616434abc980ceea53a8542 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:23:25.246905 28291 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0444894f7616434abc980ceea53a8542 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:23:25.246956 28291 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0444894f7616434abc980ceea53a8542 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:23:25.247545 28291 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0444894f7616434abc980ceea53a8542 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0444894f7616434abc980ceea53a8542" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 34437 } }
I20260812 01:23:25.247665 28291 leader_election.cc:302] T 00000000000000000000000000000000 P 0444894f7616434abc980ceea53a8542 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0444894f7616434abc980ceea53a8542; no voters:
I20260812 01:23:25.247861 28291 leader_election.cc:288] T 00000000000000000000000000000000 P 0444894f7616434abc980ceea53a8542 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 01:23:25.247886 28298 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0444894f7616434abc980ceea53a8542 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:23:25.248000 28298 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0444894f7616434abc980ceea53a8542 [term 1 LEADER]: Becoming Leader. State: Replica: 0444894f7616434abc980ceea53a8542, State: Running, Role: LEADER
I20260812 01:23:25.248170 28291 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0444894f7616434abc980ceea53a8542 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:23:25.248193 28298 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0444894f7616434abc980ceea53a8542 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0444894f7616434abc980ceea53a8542" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 34437 } }
I20260812 01:23:25.248623 28301 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0444894f7616434abc980ceea53a8542 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0444894f7616434abc980ceea53a8542" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0444894f7616434abc980ceea53a8542" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 34437 } } }
I20260812 01:23:25.248643 28299 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0444894f7616434abc980ceea53a8542 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0444894f7616434abc980ceea53a8542. Latest consensus state: current_term: 1 leader_uuid: "0444894f7616434abc980ceea53a8542" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0444894f7616434abc980ceea53a8542" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 34437 } } }
I20260812 01:23:25.248772 28299 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0444894f7616434abc980ceea53a8542 [sys.catalog]: This master's current role is: LEADER
I20260812 01:23:25.248723 28301 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0444894f7616434abc980ceea53a8542 [sys.catalog]: This master's current role is: LEADER
I20260812 01:23:25.249042 28308 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:23:25.249653 28308 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:23:25.251327 28308 catalog_manager.cc:1347] Generated new cluster ID: cd2c0fcc6882498297c258ac0112d465
I20260812 01:23:25.251382 28308 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 01:23:25.251940 28297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:25.245183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46633 (local address 127.24.0.190:34437)
0812 01:23:25.245372 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:25.245381 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:25.245408 (+ 27us) server_negotiation.cc:407] Connection header received
0812 01:23:25.246308 (+ 900us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:25.246314 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:25.246363 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:25.246480 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:23:25.246778 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:25.248243 (+ 1465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:25.249409 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:25.250147 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:25.250266 (+ 119us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:25.251133 (+ 867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:23:25.251137 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:23:25.251138 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:23:25.251399 (+ 261us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:23:25.251443 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:23:25.251459 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:23:25.251554 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:25.251643 (+ 89us) server_negotiation.cc:299] Negotiation successful
0812 01:23:25.251772 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":99,"threads_started":1}
I20260812 01:23:25.257104 24578 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 01:23:25.267402 28297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:25.264459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53639 (local address 127.24.0.190:34437)
0812 01:23:25.264509 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:25.264511 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:25.264629 (+ 118us) server_negotiation.cc:407] Connection header received
0812 01:23:25.264713 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:25.264713 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:25.264723 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:25.264790 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:23:25.264937 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:25.265421 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:25.266142 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:25.266362 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:25.266436 (+ 74us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:25.267117 (+ 681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:23:25.267118 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:23:25.267118 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:23:25.267122 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:23:25.267137 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:23:25.267141 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:23:25.267217 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:25.267284 (+ 67us) server_negotiation.cc:299] Negotiation successful
0812 01:23:25.267297 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 01:23:25.276850 28308 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:23:25.277424 28308 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:23:25.289948 28308 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0444894f7616434abc980ceea53a8542: Generated new TSK 0
I20260812 01:23:25.290210 28308 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 01:23:25.300704 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 28188
W20260812 01:23:25.322221 28164 connection.cc:535] client connection to 127.24.0.190:34437 recv error: Network error: failed to read from TLS socket (remote: 127.24.0.190:34437): Connection reset by peer (error 104)
2026-08-12T01:23:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1621 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786497534195564-24578-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 01:23:25 dist-test-slave-t3q3 krb5kdc[28343](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 01:23:25 dist-test-slave-t3q3 krb5kdc[28343](info): set up 2 sockets
Aug 12 01:23:25 dist-test-slave-t3q3 krb5kdc[28343](info): commencing operation
krb5kdc: starting...
W20260812 01:23:26.592458 24578 mini_kdc.cc:121] Time spent starting KDC: real 1.256s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786497534195564-24578-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786497534195564-24578-0/krb5kdc/test-user.keytab.
Aug 12 01:23:26 dist-test-slave-t3q3 krb5kdc[28343](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497806, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T01:23:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:23:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.0.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.0.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.0.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab.
Entry for principal kudu/127.24.0.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab.
I20260812 01:23:26.713037 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786497534195564-24578-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786497534195564-24578-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786497534195564-24578-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.190:34477
--webserver_interface=127.24.0.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab
--principal=kudu/127.24.0.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:35297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.0.190:34477
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:23:26.798770 28369 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:23:26.798978 28369 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:23:26.799001 28369 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:23:26.800088 28369 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:23:26.800122 28369 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:23:26.800136 28369 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:23:26.800148 28369 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:23:26.800160 28369 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:23:26.801561 28369 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:35297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786497534195564-24578-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786497534195564-24578-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.0.190:34477
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab
--principal=kudu/127.24.0.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.190:34477
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.0.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28369
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786497534195564-24578-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:23:26.801759 28369 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:23:26.801924 28369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:23:26.806722 28384 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:23:26.806677 28380 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:23:26.806710 28369 server_base.cc:1034] running on GCE node
W20260812 01:23:26.806677 28381 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:23:26.807312 28369 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:23:26.807626 28369 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:23:26.808787 28369 hybrid_clock.cc:648] HybridClock initialized: now 1786497806808781 us; error 37 us; skew 500 ppm
Aug 12 01:23:26 dist-test-slave-t3q3 krb5kdc[28343](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497806, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:23:26.811264 28369 init.cc:377] Logged in from keytab as kudu/127.24.0.190@KRBTEST.COM (short username kudu)
I20260812 01:23:26.811655 28369 webserver.cc:460] Webserver started at http://127.24.0.190:33607/ using document root <none> and password file <none>
I20260812 01:23:26.811911 28369 fs_manager.cc:359] Metadata directory not provided
I20260812 01:23:26.811957 28369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:23:26.812127 28369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:23:26.813235 28369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786497534195564-24578-0/minicluster-data/master-0/data/instance:
uuid: "841fa9188a464e30b5b7491f0a7838d8"
format_stamp: "Formatted at 2026-08-12 01:23:26 on dist-test-slave-t3q3"
server_key: "75e6e8f7434b9dfea4b170026f4a3dce"
server_key_iv: "fa4d6a1e595c53b80c5e8a79e4889f2c"
server_key_version: "encryptionkey@0"
I20260812 01:23:26.813604 28369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786497534195564-24578-0/minicluster-data/master-0/wal/instance:
uuid: "841fa9188a464e30b5b7491f0a7838d8"
format_stamp: "Formatted at 2026-08-12 01:23:26 on dist-test-slave-t3q3"
server_key: "75e6e8f7434b9dfea4b170026f4a3dce"
server_key_iv: "fa4d6a1e595c53b80c5e8a79e4889f2c"
server_key_version: "encryptionkey@0"
I20260812 01:23:26.815004 28369 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 01:23:26.815733 28399 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:23:26.815903 28369 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 01:23:26.816005 28369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786497534195564-24578-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786497534195564-24578-0/minicluster-data/master-0/wal
uuid: "841fa9188a464e30b5b7491f0a7838d8"
format_stamp: "Formatted at 2026-08-12 01:23:26 on dist-test-slave-t3q3"
server_key: "75e6e8f7434b9dfea4b170026f4a3dce"
server_key_iv: "fa4d6a1e595c53b80c5e8a79e4889f2c"
server_key_version: "encryptionkey@0"
I20260812 01:23:26.816118 28369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786497534195564-24578-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786497534195564-24578-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786497534195564-24578-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:23:26.819402 28369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:23:26.821293 28369 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:23:26.821404 28369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:23:26.825515 28369 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.190:34477
I20260812 01:23:26.825584 28475 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.190:34477 every 8 connection(s)
I20260812 01:23:26.825911 28369 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
I20260812 01:23:26.826948 28476 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:23:26.827821 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 28369
I20260812 01:23:26.827952 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786497534195564-24578-0/minicluster-data/master-0/wal/instance
I20260812 01:23:26.828186 24578 external_mini_cluster.cc:1428] Setting key 5fccc2dd6961b7d48e9b5a28456017e4
I20260812 01:23:26.829944 28476 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 841fa9188a464e30b5b7491f0a7838d8: Bootstrap starting.
I20260812 01:23:26.830946 28479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:26.828719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34371 (local address 127.24.0.190:34477)
0812 01:23:26.828927 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:26.828936 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:26.828971 (+ 35us) server_negotiation.cc:407] Connection header received
0812 01:23:26.829686 (+ 715us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:26.829696 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:26.829730 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:26.829895 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:23:26.830503 (+ 608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:23:26.830507 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:23:26.830507 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:23:26.830512 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:23:26.830558 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:23:26.830571 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:23:26.830686 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:26.830720 (+ 34us) server_negotiation.cc:299] Negotiation successful
0812 01:23:26.830761 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":90,"threads_started":1}
I20260812 01:23:26.831028 28476 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 841fa9188a464e30b5b7491f0a7838d8: Neither blocks nor log segments found. Creating new log.
I20260812 01:23:26.831360 28476 log.cc:824] T 00000000000000000000000000000000 P 841fa9188a464e30b5b7491f0a7838d8: Log is configured to *not* fsync() on all Append() calls
I20260812 01:23:26.832013 24578 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 01:23:26.832244 28476 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 841fa9188a464e30b5b7491f0a7838d8: No bootstrap required, opened a new log
I20260812 01:23:26.834717 28479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:26.833620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51017 (local address 127.24.0.190:34477)
0812 01:23:26.833678 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:26.833680 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:26.833823 (+ 143us) server_negotiation.cc:407] Connection header received
0812 01:23:26.833874 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:26.833875 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:26.833879 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:26.833937 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:23:26.834503 (+ 566us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:23:26.834504 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:23:26.834505 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:23:26.834510 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:23:26.834529 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:23:26.834534 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:23:26.834605 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:26.834640 (+ 35us) server_negotiation.cc:299] Negotiation successful
0812 01:23:26.834659 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 01:23:26.834865 28476 raft_consensus.cc:348] T 00000000000000000000000000000000 P 841fa9188a464e30b5b7491f0a7838d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "841fa9188a464e30b5b7491f0a7838d8" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 34477 } }
I20260812 01:23:26.835043 28476 raft_consensus.cc:374] T 00000000000000000000000000000000 P 841fa9188a464e30b5b7491f0a7838d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:23:26.835090 28476 raft_consensus.cc:729] T 00000000000000000000000000000000 P 841fa9188a464e30b5b7491f0a7838d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 841fa9188a464e30b5b7491f0a7838d8, State: Initialized, Role: FOLLOWER
I20260812 01:23:26.835237 28476 consensus_queue.cc:260] T 00000000000000000000000000000000 P 841fa9188a464e30b5b7491f0a7838d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "841fa9188a464e30b5b7491f0a7838d8" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 34477 } }
I20260812 01:23:26.835338 28476 raft_consensus.cc:388] T 00000000000000000000000000000000 P 841fa9188a464e30b5b7491f0a7838d8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:23:26.835363 28476 raft_consensus.cc:482] T 00000000000000000000000000000000 P 841fa9188a464e30b5b7491f0a7838d8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:23:26.835399 28476 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 841fa9188a464e30b5b7491f0a7838d8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:23:26.835924 28476 raft_consensus.cc:504] T 00000000000000000000000000000000 P 841fa9188a464e30b5b7491f0a7838d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "841fa9188a464e30b5b7491f0a7838d8" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 34477 } }
I20260812 01:23:26.836015 28476 leader_election.cc:302] T 00000000000000000000000000000000 P 841fa9188a464e30b5b7491f0a7838d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 841fa9188a464e30b5b7491f0a7838d8; no voters:
I20260812 01:23:26.836199 28476 leader_election.cc:288] T 00000000000000000000000000000000 P 841fa9188a464e30b5b7491f0a7838d8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 01:23:26.836220 28498 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 841fa9188a464e30b5b7491f0a7838d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:23:26.836338 28498 raft_consensus.cc:686] T 00000000000000000000000000000000 P 841fa9188a464e30b5b7491f0a7838d8 [term 1 LEADER]: Becoming Leader. State: Replica: 841fa9188a464e30b5b7491f0a7838d8, State: Running, Role: LEADER
I20260812 01:23:26.836426 28498 consensus_queue.cc:237] T 00000000000000000000000000000000 P 841fa9188a464e30b5b7491f0a7838d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "841fa9188a464e30b5b7491f0a7838d8" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 34477 } }
I20260812 01:23:26.836465 28476 sys_catalog.cc:564] T 00000000000000000000000000000000 P 841fa9188a464e30b5b7491f0a7838d8 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:23:26.836824 28499 sys_catalog.cc:455] T 00000000000000000000000000000000 P 841fa9188a464e30b5b7491f0a7838d8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "841fa9188a464e30b5b7491f0a7838d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "841fa9188a464e30b5b7491f0a7838d8" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 34477 } } }
I20260812 01:23:26.836941 28499 sys_catalog.cc:458] T 00000000000000000000000000000000 P 841fa9188a464e30b5b7491f0a7838d8 [sys.catalog]: This master's current role is: LEADER
I20260812 01:23:26.837133 28501 sys_catalog.cc:455] T 00000000000000000000000000000000 P 841fa9188a464e30b5b7491f0a7838d8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 841fa9188a464e30b5b7491f0a7838d8. Latest consensus state: current_term: 1 leader_uuid: "841fa9188a464e30b5b7491f0a7838d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "841fa9188a464e30b5b7491f0a7838d8" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 34477 } } }
I20260812 01:23:26.837200 28501 sys_catalog.cc:458] T 00000000000000000000000000000000 P 841fa9188a464e30b5b7491f0a7838d8 [sys.catalog]: This master's current role is: LEADER
I20260812 01:23:26.837550 28506 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:23:26.838040 28506 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:23:26.839628 28506 catalog_manager.cc:1347] Generated new cluster ID: 5eb225ffbfdf41a1a2fa6b7f4ae357d6
I20260812 01:23:26.839679 28506 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 01:23:26.853749 28506 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:23:26.853950 28506 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:23:26.856647 28506 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 841fa9188a464e30b5b7491f0a7838d8: Generated new TSK 0
I20260812 01:23:26.856827 28506 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 01:23:26.862690 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 28369
2026-08-12T01:23:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1551 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786497534195564-24578-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 01:23:26 dist-test-slave-t3q3 krb5kdc[28534](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 01:23:26 dist-test-slave-t3q3 krb5kdc[28534](info): set up 2 sockets
Aug 12 01:23:26 dist-test-slave-t3q3 krb5kdc[28534](info): commencing operation
krb5kdc: starting...
W20260812 01:23:28.128681 24578 mini_kdc.cc:121] Time spent starting KDC: real 1.241s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786497534195564-24578-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786497534195564-24578-0/krb5kdc/test-user.keytab.
Aug 12 01:23:28 dist-test-slave-t3q3 krb5kdc[28534](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497808, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T01:23:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:23:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.0.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.0.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.0.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab.
Entry for principal kudu/127.24.0.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab.
I20260812 01:23:28.254644 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786497534195564-24578-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786497534195564-24578-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786497534195564-24578-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.190:43795
--webserver_interface=127.24.0.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab
--principal=kudu/127.24.0.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:38961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.0.190:43795
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:23:28.337188 28556 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:23:28.337392 28556 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:23:28.337416 28556 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:23:28.338485 28556 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:23:28.338519 28556 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:23:28.338534 28556 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:23:28.338547 28556 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:23:28.338558 28556 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:23:28.339964 28556 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:38961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786497534195564-24578-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786497534195564-24578-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.0.190:43795
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab
--principal=kudu/127.24.0.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.190:43795
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.0.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28556
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786497534195564-24578-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:23:28.340227 28556 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:23:28.340422 28556 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:23:28.345291 28563 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:23:28.345270 28567 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:23:28.345450 28556 server_base.cc:1034] running on GCE node
W20260812 01:23:28.345257 28564 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:23:28.345880 28556 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:23:28.346203 28556 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:23:28.347389 28556 hybrid_clock.cc:648] HybridClock initialized: now 1786497808347369 us; error 30 us; skew 500 ppm
Aug 12 01:23:28 dist-test-slave-t3q3 krb5kdc[28534](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497808, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:23:28.353834 28556 init.cc:377] Logged in from keytab as kudu/127.24.0.190@KRBTEST.COM (short username kudu)
I20260812 01:23:28.354332 28556 webserver.cc:460] Webserver started at http://127.24.0.190:35841/ using document root <none> and password file <none>
I20260812 01:23:28.354591 28556 fs_manager.cc:359] Metadata directory not provided
I20260812 01:23:28.354658 28556 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:23:28.354804 28556 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:23:28.355896 28556 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786497534195564-24578-0/minicluster-data/master-0/data/instance:
uuid: "4242441e96964a6098014cc2ca5f36d7"
format_stamp: "Formatted at 2026-08-12 01:23:28 on dist-test-slave-t3q3"
server_key: "da64ddcb51cf07445a673756eff42255"
server_key_iv: "f41273808316a21cc993ee2eae82267e"
server_key_version: "encryptionkey@0"
I20260812 01:23:28.356299 28556 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786497534195564-24578-0/minicluster-data/master-0/wal/instance:
uuid: "4242441e96964a6098014cc2ca5f36d7"
format_stamp: "Formatted at 2026-08-12 01:23:28 on dist-test-slave-t3q3"
server_key: "da64ddcb51cf07445a673756eff42255"
server_key_iv: "f41273808316a21cc993ee2eae82267e"
server_key_version: "encryptionkey@0"
I20260812 01:23:28.357656 28556 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 01:23:28.358395 28579 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:23:28.358688 28556 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 01:23:28.358793 28556 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786497534195564-24578-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786497534195564-24578-0/minicluster-data/master-0/wal
uuid: "4242441e96964a6098014cc2ca5f36d7"
format_stamp: "Formatted at 2026-08-12 01:23:28 on dist-test-slave-t3q3"
server_key: "da64ddcb51cf07445a673756eff42255"
server_key_iv: "f41273808316a21cc993ee2eae82267e"
server_key_version: "encryptionkey@0"
I20260812 01:23:28.358865 28556 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786497534195564-24578-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786497534195564-24578-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786497534195564-24578-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:23:28.382974 28556 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:23:28.385546 28556 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:23:28.385720 28556 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:23:28.390021 28556 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.190:43795
I20260812 01:23:28.390059 28664 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.190:43795 every 8 connection(s)
I20260812 01:23:28.390479 28556 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
I20260812 01:23:28.391433 28665 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:23:28.394100 28665 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4242441e96964a6098014cc2ca5f36d7: Bootstrap starting.
I20260812 01:23:28.394927 28665 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4242441e96964a6098014cc2ca5f36d7: Neither blocks nor log segments found. Creating new log.
I20260812 01:23:28.395318 28665 log.cc:824] T 00000000000000000000000000000000 P 4242441e96964a6098014cc2ca5f36d7: Log is configured to *not* fsync() on all Append() calls
I20260812 01:23:28.396152 28665 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4242441e96964a6098014cc2ca5f36d7: No bootstrap required, opened a new log
I20260812 01:23:28.397826 28665 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4242441e96964a6098014cc2ca5f36d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4242441e96964a6098014cc2ca5f36d7" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 43795 } }
I20260812 01:23:28.397975 28665 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4242441e96964a6098014cc2ca5f36d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:23:28.398000 28665 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4242441e96964a6098014cc2ca5f36d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4242441e96964a6098014cc2ca5f36d7, State: Initialized, Role: FOLLOWER
I20260812 01:23:28.398149 28665 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4242441e96964a6098014cc2ca5f36d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4242441e96964a6098014cc2ca5f36d7" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 43795 } }
I20260812 01:23:28.398232 28665 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4242441e96964a6098014cc2ca5f36d7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:23:28.398255 28665 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4242441e96964a6098014cc2ca5f36d7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:23:28.398324 28665 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4242441e96964a6098014cc2ca5f36d7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:23:28.399235 28665 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4242441e96964a6098014cc2ca5f36d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4242441e96964a6098014cc2ca5f36d7" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 43795 } }
I20260812 01:23:28.399353 28665 leader_election.cc:302] T 00000000000000000000000000000000 P 4242441e96964a6098014cc2ca5f36d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4242441e96964a6098014cc2ca5f36d7; no voters:
I20260812 01:23:28.399529 28665 leader_election.cc:288] T 00000000000000000000000000000000 P 4242441e96964a6098014cc2ca5f36d7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 01:23:28.399566 28668 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4242441e96964a6098014cc2ca5f36d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:23:28.399796 28668 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4242441e96964a6098014cc2ca5f36d7 [term 1 LEADER]: Becoming Leader. State: Replica: 4242441e96964a6098014cc2ca5f36d7, State: Running, Role: LEADER
I20260812 01:23:28.399847 28665 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4242441e96964a6098014cc2ca5f36d7 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:23:28.399897 28668 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4242441e96964a6098014cc2ca5f36d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4242441e96964a6098014cc2ca5f36d7" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 43795 } }
I20260812 01:23:28.400306 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 28556
I20260812 01:23:28.400400 28670 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4242441e96964a6098014cc2ca5f36d7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4242441e96964a6098014cc2ca5f36d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4242441e96964a6098014cc2ca5f36d7" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 43795 } } }
I20260812 01:23:28.400455 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786497534195564-24578-0/minicluster-data/master-0/wal/instance
I20260812 01:23:28.400485 28670 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4242441e96964a6098014cc2ca5f36d7 [sys.catalog]: This master's current role is: LEADER
I20260812 01:23:28.400701 24578 external_mini_cluster.cc:1428] Setting key f04ef7e17be52d6e704d1d7cc5de087f
I20260812 01:23:28.400727 28688 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:23:28.401245 28677 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4242441e96964a6098014cc2ca5f36d7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4242441e96964a6098014cc2ca5f36d7. Latest consensus state: current_term: 1 leader_uuid: "4242441e96964a6098014cc2ca5f36d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4242441e96964a6098014cc2ca5f36d7" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 43795 } } }
I20260812 01:23:28.401566 28688 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:23:28.401646 28677 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4242441e96964a6098014cc2ca5f36d7 [sys.catalog]: This master's current role is: LEADER
I20260812 01:23:28.403681 28688 catalog_manager.cc:1347] Generated new cluster ID: 651664573eff442bb9b968ff079db9bc
I20260812 01:23:28.403743 28688 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 01:23:28 dist-test-slave-t3q3 krb5kdc[28534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497808, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
I20260812 01:23:28.413755 28707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:28.401457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47663 (local address 127.24.0.190:43795)
0812 01:23:28.401836 (+ 379us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:28.401848 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:28.401884 (+ 36us) server_negotiation.cc:407] Connection header received
0812 01:23:28.402019 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:28.402027 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:28.402089 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:28.402248 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:23:28.403271 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:28.404842 (+ 1571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:28.405968 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:28.406654 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:28.406760 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:28.409842 (+ 3082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:23:28.409852 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:23:28.409852 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:23:28.409865 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:23:28.411996 (+ 2131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:23:28.412425 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:23:28.412429 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:23:28.412430 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:23:28.412560 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:23:28.412768 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:23:28.412769 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:23:28.412769 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:23:28.413113 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:23:28.413295 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:28.413367 (+ 72us) server_negotiation.cc:299] Negotiation successful
0812 01:23:28.413512 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":121,"threads_started":1}
I20260812 01:23:28.414784 24578 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 01:23:28.418404 28688 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:23:28.419135 28688 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:23:28.423859 28707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:28.416345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52093 (local address 127.24.0.190:43795)
0812 01:23:28.416380 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:28.416382 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:28.416444 (+ 62us) server_negotiation.cc:407] Connection header received
0812 01:23:28.416517 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:28.416518 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:28.416537 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:28.416600 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:23:28.417284 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:28.418043 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:28.419140 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:28.419480 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:28.419553 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:28.421020 (+ 1467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:23:28.421021 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:23:28.421022 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:23:28.421029 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:23:28.422688 (+ 1659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:23:28.423054 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:23:28.423056 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:23:28.423056 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:23:28.423118 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:23:28.423316 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:23:28.423317 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:23:28.423317 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:23:28.423446 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:23:28.423544 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:28.423675 (+ 131us) server_negotiation.cc:299] Negotiation successful
0812 01:23:28.423771 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260812 01:23:28.426105 28688 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4242441e96964a6098014cc2ca5f36d7: Generated new TSK 0
I20260812 01:23:28.426307 28688 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 01:23:28.428380 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 28556
2026-08-12T01:23:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1567 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786497534195564-24578-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 01:23:28 dist-test-slave-t3q3 krb5kdc[28720](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 01:23:28 dist-test-slave-t3q3 krb5kdc[28720](info): set up 2 sockets
Aug 12 01:23:28 dist-test-slave-t3q3 krb5kdc[28720](info): commencing operation
krb5kdc: starting...
W20260812 01:23:29.700989 24578 mini_kdc.cc:121] Time spent starting KDC: real 1.247s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786497534195564-24578-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786497534195564-24578-0/krb5kdc/test-user.keytab.
Aug 12 01:23:29 dist-test-slave-t3q3 krb5kdc[28720](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497809, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T01:23:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:23:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.0.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.0.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.0.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab.
Entry for principal kudu/127.24.0.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab.
I20260812 01:23:29.824406 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786497534195564-24578-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786497534195564-24578-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786497534195564-24578-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.190:40925
--webserver_interface=127.24.0.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab
--principal=kudu/127.24.0.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:43019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.0.190:40925
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:23:29.907128 28742 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:23:29.907368 28742 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:23:29.907407 28742 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:23:29.908577 28742 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:23:29.908644 28742 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:23:29.908668 28742 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:23:29.908689 28742 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:23:29.908717 28742 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:23:29.910143 28742 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:43019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786497534195564-24578-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786497534195564-24578-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.0.190:40925
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab
--principal=kudu/127.24.0.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.190:40925
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.0.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28742
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786497534195564-24578-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:23:29.910409 28742 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:23:29.910629 28742 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:23:29.915400 28754 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:23:29.915422 28756 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:23:29.915433 28742 server_base.cc:1034] running on GCE node
W20260812 01:23:29.915403 28753 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:23:29.915928 28742 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:23:29.916246 28742 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:23:29.917411 28742 hybrid_clock.cc:648] HybridClock initialized: now 1786497809917400 us; error 40 us; skew 500 ppm
Aug 12 01:23:29 dist-test-slave-t3q3 krb5kdc[28720](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497809, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:23:29.919943 28742 init.cc:377] Logged in from keytab as kudu/127.24.0.190@KRBTEST.COM (short username kudu)
I20260812 01:23:29.920384 28742 webserver.cc:460] Webserver started at http://127.24.0.190:35013/ using document root <none> and password file <none>
I20260812 01:23:29.920564 28742 fs_manager.cc:359] Metadata directory not provided
I20260812 01:23:29.920603 28742 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:23:29.920697 28742 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:23:29.921643 28742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786497534195564-24578-0/minicluster-data/master-0/data/instance:
uuid: "5dc32468ddff4b9ebe9a59c1caf94485"
format_stamp: "Formatted at 2026-08-12 01:23:29 on dist-test-slave-t3q3"
server_key: "a294bcfe32f4ddbbe115324afaab91e4"
server_key_iv: "4777e33cd98e77483b7c919dfd18729c"
server_key_version: "encryptionkey@0"
I20260812 01:23:29.921962 28742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786497534195564-24578-0/minicluster-data/master-0/wal/instance:
uuid: "5dc32468ddff4b9ebe9a59c1caf94485"
format_stamp: "Formatted at 2026-08-12 01:23:29 on dist-test-slave-t3q3"
server_key: "a294bcfe32f4ddbbe115324afaab91e4"
server_key_iv: "4777e33cd98e77483b7c919dfd18729c"
server_key_version: "encryptionkey@0"
I20260812 01:23:29.923221 28742 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 01:23:29.923935 28765 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:23:29.924110 28742 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:23:29.924204 28742 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786497534195564-24578-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786497534195564-24578-0/minicluster-data/master-0/wal
uuid: "5dc32468ddff4b9ebe9a59c1caf94485"
format_stamp: "Formatted at 2026-08-12 01:23:29 on dist-test-slave-t3q3"
server_key: "a294bcfe32f4ddbbe115324afaab91e4"
server_key_iv: "4777e33cd98e77483b7c919dfd18729c"
server_key_version: "encryptionkey@0"
I20260812 01:23:29.924261 28742 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786497534195564-24578-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786497534195564-24578-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786497534195564-24578-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:23:29.934849 28742 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:23:29.936806 28742 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:23:29.936913 28742 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:23:29.940757 28742 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.190:40925
I20260812 01:23:29.940812 28849 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.190:40925 every 8 connection(s)
I20260812 01:23:29.941126 28742 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
I20260812 01:23:29.941979 28851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:23:29.944343 28851 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5dc32468ddff4b9ebe9a59c1caf94485: Bootstrap starting.
I20260812 01:23:29.944937 28851 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5dc32468ddff4b9ebe9a59c1caf94485: Neither blocks nor log segments found. Creating new log.
I20260812 01:23:29.945235 28851 log.cc:824] T 00000000000000000000000000000000 P 5dc32468ddff4b9ebe9a59c1caf94485: Log is configured to *not* fsync() on all Append() calls
I20260812 01:23:29.945933 28851 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5dc32468ddff4b9ebe9a59c1caf94485: No bootstrap required, opened a new log
I20260812 01:23:29.947538 28851 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5dc32468ddff4b9ebe9a59c1caf94485 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5dc32468ddff4b9ebe9a59c1caf94485" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 40925 } }
I20260812 01:23:29.947700 28851 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5dc32468ddff4b9ebe9a59c1caf94485 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:23:29.947748 28851 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5dc32468ddff4b9ebe9a59c1caf94485 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5dc32468ddff4b9ebe9a59c1caf94485, State: Initialized, Role: FOLLOWER
I20260812 01:23:29.947927 28851 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5dc32468ddff4b9ebe9a59c1caf94485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5dc32468ddff4b9ebe9a59c1caf94485" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 40925 } }
I20260812 01:23:29.948028 28851 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5dc32468ddff4b9ebe9a59c1caf94485 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:23:29.948094 28851 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5dc32468ddff4b9ebe9a59c1caf94485 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:23:29.948145 28851 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5dc32468ddff4b9ebe9a59c1caf94485 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:23:29.949060 28851 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5dc32468ddff4b9ebe9a59c1caf94485 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5dc32468ddff4b9ebe9a59c1caf94485" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 40925 } }
I20260812 01:23:29.949115 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 28742
I20260812 01:23:29.949218 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786497534195564-24578-0/minicluster-data/master-0/wal/instance
I20260812 01:23:29.949201 28851 leader_election.cc:302] T 00000000000000000000000000000000 P 5dc32468ddff4b9ebe9a59c1caf94485 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5dc32468ddff4b9ebe9a59c1caf94485; no voters:
I20260812 01:23:29.949369 28851 leader_election.cc:288] T 00000000000000000000000000000000 P 5dc32468ddff4b9ebe9a59c1caf94485 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 01:23:29.949420 28856 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5dc32468ddff4b9ebe9a59c1caf94485 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:23:29.949458 24578 external_mini_cluster.cc:1428] Setting key 88be96d418def791cb3f1860d081bbce
I20260812 01:23:29.949599 28856 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5dc32468ddff4b9ebe9a59c1caf94485 [term 1 LEADER]: Becoming Leader. State: Replica: 5dc32468ddff4b9ebe9a59c1caf94485, State: Running, Role: LEADER
I20260812 01:23:29.949666 28851 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5dc32468ddff4b9ebe9a59c1caf94485 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:23:29.949726 28856 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5dc32468ddff4b9ebe9a59c1caf94485 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5dc32468ddff4b9ebe9a59c1caf94485" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 40925 } }
I20260812 01:23:29.950423 28858 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5dc32468ddff4b9ebe9a59c1caf94485 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5dc32468ddff4b9ebe9a59c1caf94485. Latest consensus state: current_term: 1 leader_uuid: "5dc32468ddff4b9ebe9a59c1caf94485" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5dc32468ddff4b9ebe9a59c1caf94485" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 40925 } } }
I20260812 01:23:29.950502 28858 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5dc32468ddff4b9ebe9a59c1caf94485 [sys.catalog]: This master's current role is: LEADER
I20260812 01:23:29.950832 28857 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5dc32468ddff4b9ebe9a59c1caf94485 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5dc32468ddff4b9ebe9a59c1caf94485" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5dc32468ddff4b9ebe9a59c1caf94485" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 40925 } } }
I20260812 01:23:29.950922 28857 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5dc32468ddff4b9ebe9a59c1caf94485 [sys.catalog]: This master's current role is: LEADER
I20260812 01:23:29.951272 28874 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:23:29.951803 28874 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:23:29.953343 28874 catalog_manager.cc:1347] Generated new cluster ID: 14b782c34dfe450b8092f4a785e00384
I20260812 01:23:29.953401 28874 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 01:23:29.955338 28861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:29.949911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42123 (local address 127.24.0.190:40925)
0812 01:23:29.950075 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:29.950084 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:29.950150 (+ 66us) server_negotiation.cc:407] Connection header received
0812 01:23:29.950336 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:29.950344 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:29.950411 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:29.950853 (+ 442us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:23:29.950976 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:29.952515 (+ 1539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:29.953518 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:29.953904 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:29.954022 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:29.954719 (+ 697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:23:29.954723 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:23:29.954723 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:23:29.954902 (+ 179us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:23:29.954934 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:23:29.954945 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:23:29.955013 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:29.955104 (+ 91us) server_negotiation.cc:299] Negotiation successful
0812 01:23:29.955167 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":80,"threads_started":1}
I20260812 01:23:29.956244 24578 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 01:23:29.966737 28874 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:23:29.967192 28861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:29.960357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54241 (local address 127.24.0.190:40925)
0812 01:23:29.960410 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:29.960412 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:29.964244 (+ 3832us) server_negotiation.cc:407] Connection header received
0812 01:23:29.964322 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:29.964322 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:29.964334 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:29.964401 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:23:29.964551 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:29.965054 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:29.965766 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:29.965969 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:29.966080 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:29.966930 (+ 850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:23:29.966932 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:23:29.966932 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:23:29.966939 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:23:29.966954 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:23:29.966959 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:23:29.967025 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:29.967090 (+ 65us) server_negotiation.cc:299] Negotiation successful
0812 01:23:29.967116 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260812 01:23:29.967403 28874 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:23:29.972544 28874 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5dc32468ddff4b9ebe9a59c1caf94485: Generated new TSK 0
I20260812 01:23:29.972728 28874 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 01:23:29.974807 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 28742
2026-08-12T01:23:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1550 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786497534195564-24578-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 01:23:30 dist-test-slave-t3q3 krb5kdc[28905](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 01:23:30 dist-test-slave-t3q3 krb5kdc[28905](info): set up 2 sockets
Aug 12 01:23:30 dist-test-slave-t3q3 krb5kdc[28905](info): commencing operation
krb5kdc: starting...
W20260812 01:23:31.265257 24578 mini_kdc.cc:121] Time spent starting KDC: real 1.260s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786497534195564-24578-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786497534195564-24578-0/krb5kdc/test-user.keytab.
Aug 12 01:23:31 dist-test-slave-t3q3 krb5kdc[28905](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497811, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T01:23:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:23:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.0.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.0.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.0.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab.
Entry for principal kudu/127.24.0.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab.
I20260812 01:23:31.395277 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786497534195564-24578-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786497534195564-24578-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786497534195564-24578-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.190:34853
--webserver_interface=127.24.0.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab
--principal=kudu/127.24.0.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:33635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.0.190:34853
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:23:31.481911 28923 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:23:31.482268 28923 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:23:31.482308 28923 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:23:31.484305 28923 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:23:31.484366 28923 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:23:31.484390 28923 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:23:31.484464 28923 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:23:31.484495 28923 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:23:31.487483 28923 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:33635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786497534195564-24578-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786497534195564-24578-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.0.190:34853
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab
--principal=kudu/127.24.0.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.190:34853
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.0.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28923
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786497534195564-24578-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:23:31.488163 28923 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:23:31.488420 28923 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:23:31.494633 28933 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:23:31.494751 28935 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:23:31.494634 28932 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:23:31.494917 28923 server_base.cc:1034] running on GCE node
I20260812 01:23:31.495222 28923 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:23:31.495558 28923 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:23:31.496771 28923 hybrid_clock.cc:648] HybridClock initialized: now 1786497811496754 us; error 27 us; skew 500 ppm
Aug 12 01:23:31 dist-test-slave-t3q3 krb5kdc[28905](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497811, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:23:31.499377 28923 init.cc:377] Logged in from keytab as kudu/127.24.0.190@KRBTEST.COM (short username kudu)
I20260812 01:23:31.499854 28923 webserver.cc:460] Webserver started at http://127.24.0.190:44751/ using document root <none> and password file <none>
I20260812 01:23:31.500200 28923 fs_manager.cc:359] Metadata directory not provided
I20260812 01:23:31.500273 28923 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:23:31.500440 28923 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:23:31.501535 28923 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786497534195564-24578-0/minicluster-data/master-0/data/instance:
uuid: "a7e2f83cf05a41ce8c68fdc7539d542b"
format_stamp: "Formatted at 2026-08-12 01:23:31 on dist-test-slave-t3q3"
server_key: "3ea9778df7572e1687c6fed571c151f4"
server_key_iv: "39ec5faa5ae74a23c732511bc9631cd3"
server_key_version: "encryptionkey@0"
I20260812 01:23:31.501931 28923 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786497534195564-24578-0/minicluster-data/master-0/wal/instance:
uuid: "a7e2f83cf05a41ce8c68fdc7539d542b"
format_stamp: "Formatted at 2026-08-12 01:23:31 on dist-test-slave-t3q3"
server_key: "3ea9778df7572e1687c6fed571c151f4"
server_key_iv: "39ec5faa5ae74a23c732511bc9631cd3"
server_key_version: "encryptionkey@0"
I20260812 01:23:31.503319 28923 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 01:23:31.504128 28945 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:23:31.504325 28923 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:23:31.504418 28923 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786497534195564-24578-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786497534195564-24578-0/minicluster-data/master-0/wal
uuid: "a7e2f83cf05a41ce8c68fdc7539d542b"
format_stamp: "Formatted at 2026-08-12 01:23:31 on dist-test-slave-t3q3"
server_key: "3ea9778df7572e1687c6fed571c151f4"
server_key_iv: "39ec5faa5ae74a23c732511bc9631cd3"
server_key_version: "encryptionkey@0"
I20260812 01:23:31.504474 28923 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786497534195564-24578-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786497534195564-24578-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786497534195564-24578-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:23:31.507720 28923 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:23:31.509625 28923 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:23:31.509732 28923 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:23:31.513711 28923 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.190:34853
I20260812 01:23:31.513801 29034 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.190:34853 every 8 connection(s)
I20260812 01:23:31.514103 28923 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
I20260812 01:23:31.515113 29036 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:23:31.517467 29036 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7e2f83cf05a41ce8c68fdc7539d542b: Bootstrap starting.
I20260812 01:23:31.518072 29036 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a7e2f83cf05a41ce8c68fdc7539d542b: Neither blocks nor log segments found. Creating new log.
I20260812 01:23:31.518388 29036 log.cc:824] T 00000000000000000000000000000000 P a7e2f83cf05a41ce8c68fdc7539d542b: Log is configured to *not* fsync() on all Append() calls
I20260812 01:23:31.519110 29036 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7e2f83cf05a41ce8c68fdc7539d542b: No bootstrap required, opened a new log
I20260812 01:23:31.520862 29036 raft_consensus.cc:348] T 00000000000000000000000000000000 P a7e2f83cf05a41ce8c68fdc7539d542b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7e2f83cf05a41ce8c68fdc7539d542b" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 34853 } }
I20260812 01:23:31.521030 29036 raft_consensus.cc:374] T 00000000000000000000000000000000 P a7e2f83cf05a41ce8c68fdc7539d542b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:23:31.521078 29036 raft_consensus.cc:729] T 00000000000000000000000000000000 P a7e2f83cf05a41ce8c68fdc7539d542b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7e2f83cf05a41ce8c68fdc7539d542b, State: Initialized, Role: FOLLOWER
I20260812 01:23:31.521276 29036 consensus_queue.cc:260] T 00000000000000000000000000000000 P a7e2f83cf05a41ce8c68fdc7539d542b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7e2f83cf05a41ce8c68fdc7539d542b" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 34853 } }
I20260812 01:23:31.521384 29036 raft_consensus.cc:388] T 00000000000000000000000000000000 P a7e2f83cf05a41ce8c68fdc7539d542b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:23:31.521430 29036 raft_consensus.cc:482] T 00000000000000000000000000000000 P a7e2f83cf05a41ce8c68fdc7539d542b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:23:31.521486 29036 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a7e2f83cf05a41ce8c68fdc7539d542b [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:23:31.522346 29036 raft_consensus.cc:504] T 00000000000000000000000000000000 P a7e2f83cf05a41ce8c68fdc7539d542b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7e2f83cf05a41ce8c68fdc7539d542b" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 34853 } }
I20260812 01:23:31.522477 29036 leader_election.cc:302] T 00000000000000000000000000000000 P a7e2f83cf05a41ce8c68fdc7539d542b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a7e2f83cf05a41ce8c68fdc7539d542b; no voters:
I20260812 01:23:31.522677 29036 leader_election.cc:288] T 00000000000000000000000000000000 P a7e2f83cf05a41ce8c68fdc7539d542b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 01:23:31.522718 29042 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a7e2f83cf05a41ce8c68fdc7539d542b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:23:31.522904 29042 raft_consensus.cc:686] T 00000000000000000000000000000000 P a7e2f83cf05a41ce8c68fdc7539d542b [term 1 LEADER]: Becoming Leader. State: Replica: a7e2f83cf05a41ce8c68fdc7539d542b, State: Running, Role: LEADER
I20260812 01:23:31.522970 29036 sys_catalog.cc:564] T 00000000000000000000000000000000 P a7e2f83cf05a41ce8c68fdc7539d542b [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:23:31.523078 29042 consensus_queue.cc:237] T 00000000000000000000000000000000 P a7e2f83cf05a41ce8c68fdc7539d542b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7e2f83cf05a41ce8c68fdc7539d542b" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 34853 } }
I20260812 01:23:31.523201 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 28923
I20260812 01:23:31.523325 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786497534195564-24578-0/minicluster-data/master-0/wal/instance
I20260812 01:23:31.523540 24578 external_mini_cluster.cc:1428] Setting key 14835da7dd7d043cadecd4ff5beb7bde
I20260812 01:23:31.523756 29043 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7e2f83cf05a41ce8c68fdc7539d542b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a7e2f83cf05a41ce8c68fdc7539d542b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7e2f83cf05a41ce8c68fdc7539d542b" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 34853 } } }
I20260812 01:23:31.523775 29044 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7e2f83cf05a41ce8c68fdc7539d542b [sys.catalog]: SysCatalogTable state changed. Reason: New leader a7e2f83cf05a41ce8c68fdc7539d542b. Latest consensus state: current_term: 1 leader_uuid: "a7e2f83cf05a41ce8c68fdc7539d542b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7e2f83cf05a41ce8c68fdc7539d542b" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 34853 } } }
I20260812 01:23:31.523841 29043 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7e2f83cf05a41ce8c68fdc7539d542b [sys.catalog]: This master's current role is: LEADER
I20260812 01:23:31.523871 29044 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7e2f83cf05a41ce8c68fdc7539d542b [sys.catalog]: This master's current role is: LEADER
I20260812 01:23:31.524317 29055 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:23:31.524873 29055 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:23:31.526432 29057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:31.524167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44979 (local address 127.24.0.190:34853)
0812 01:23:31.524727 (+ 560us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:31.524737 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:31.524774 (+ 37us) server_negotiation.cc:407] Connection header received
0812 01:23:31.524893 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:31.524900 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:31.524930 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:31.525164 (+ 234us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:23:31.525843 (+ 679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:23:31.525847 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:23:31.525847 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:23:31.525852 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:23:31.525888 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:23:31.525903 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:23:31.525967 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:31.526206 (+ 239us) server_negotiation.cc:299] Negotiation successful
0812 01:23:31.526246 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":501,"thread_start_us":76,"threads_started":1}
I20260812 01:23:31.526937 29055 catalog_manager.cc:1347] Generated new cluster ID: fea73fb63b034aac840a2191d7a96f59
I20260812 01:23:31.527104 29055 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 01:23:31.527410 24578 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 01:23:31.529888 29057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:31.528800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49295 (local address 127.24.0.190:34853)
0812 01:23:31.528834 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:31.528836 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:31.528983 (+ 147us) server_negotiation.cc:407] Connection header received
0812 01:23:31.529032 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:31.529033 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:31.529040 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:31.529097 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:23:31.529691 (+ 594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:23:31.529691 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:23:31.529692 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:23:31.529695 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:23:31.529706 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:23:31.529709 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:23:31.529756 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:31.529812 (+ 56us) server_negotiation.cc:299] Negotiation successful
0812 01:23:31.529823 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 01:23:31.534871 29055 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:23:31.535030 29055 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:23:31.538882 29055 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a7e2f83cf05a41ce8c68fdc7539d542b: Generated new TSK 0
I20260812 01:23:31.539044 29055 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 01:23:31.543429 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 28923
2026-08-12T01:23:31Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1565 ms)
[----------] 18 tests from AuthTokenIssuingTest (28320 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786497534195564-24578-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 01:23:31 dist-test-slave-t3q3 krb5kdc[29087](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 01:23:31 dist-test-slave-t3q3 krb5kdc[29087](info): set up 2 sockets
Aug 12 01:23:31 dist-test-slave-t3q3 krb5kdc[29087](info): commencing operation
krb5kdc: starting...
W20260812 01:23:32.837111 24578 mini_kdc.cc:121] Time spent starting KDC: real 1.266s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786497534195564-24578-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786497534195564-24578-0/krb5kdc/test-user.keytab.
Aug 12 01:23:32 dist-test-slave-t3q3 krb5kdc[29087](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497812, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T01:23:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:23:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.0.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.0.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.0.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab.
Entry for principal kudu/127.24.0.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab.
I20260812 01:23:32.967701 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786497534195564-24578-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786497534195564-24578-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786497534195564-24578-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.190:36549
--webserver_interface=127.24.0.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab
--principal=kudu/127.24.0.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:46113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.0.190:36549,127.24.0.189:42977,127.24.0.188:33185
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:23:33.052256 29109 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 01:23:33.052536 29109 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:23:33.052568 29109 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:23:33.052605 29109 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:23:33.053897 29109 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:23:33.054046 29109 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:23:33.054075 29109 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:23:33.054105 29109 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:23:33.054124 29109 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:23:33.055750 29109 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:46113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786497534195564-24578-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786497534195564-24578-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.0.190:36549,127.24.0.189:42977,127.24.0.188:33185
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab
--principal=kudu/127.24.0.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.190:36549
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.0.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29109
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786497534195564-24578-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:23:33.056048 29109 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:23:33.056346 29109 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:23:33.061360 29116 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:23:33.061365 29123 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:23:33.061374 29118 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:23:33.061420 29109 server_base.cc:1034] running on GCE node
I20260812 01:23:33.061897 29109 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:23:33.062220 29109 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:23:33.063387 29109 hybrid_clock.cc:648] HybridClock initialized: now 1786497813063384 us; error 41 us; skew 500 ppm
Aug 12 01:23:33 dist-test-slave-t3q3 krb5kdc[29087](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497813, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:23:33.066039 29109 init.cc:377] Logged in from keytab as kudu/127.24.0.190@KRBTEST.COM (short username kudu)
I20260812 01:23:33.066483 29109 webserver.cc:460] Webserver started at http://127.24.0.190:44507/ using document root <none> and password file <none>
I20260812 01:23:33.066732 29109 fs_manager.cc:359] Metadata directory not provided
I20260812 01:23:33.066809 29109 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:23:33.066967 29109 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:23:33.068038 29109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786497534195564-24578-0/minicluster-data/master-0/data/instance:
uuid: "83c4efc95926420d8a7bf318f9dafd8f"
format_stamp: "Formatted at 2026-08-12 01:23:33 on dist-test-slave-t3q3"
server_key: "dcb317de7f13cb5ed50c78a4df404dc9"
server_key_iv: "5329e1203e2b4f40b89cdfa1267d8459"
server_key_version: "encryptionkey@0"
I20260812 01:23:33.068495 29109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786497534195564-24578-0/minicluster-data/master-0/wal/instance:
uuid: "83c4efc95926420d8a7bf318f9dafd8f"
format_stamp: "Formatted at 2026-08-12 01:23:33 on dist-test-slave-t3q3"
server_key: "dcb317de7f13cb5ed50c78a4df404dc9"
server_key_iv: "5329e1203e2b4f40b89cdfa1267d8459"
server_key_version: "encryptionkey@0"
I20260812 01:23:33.070108 29109 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 01:23:33.070925 29131 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:23:33.071141 29109 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:23:33.071255 29109 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786497534195564-24578-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786497534195564-24578-0/minicluster-data/master-0/wal
uuid: "83c4efc95926420d8a7bf318f9dafd8f"
format_stamp: "Formatted at 2026-08-12 01:23:33 on dist-test-slave-t3q3"
server_key: "dcb317de7f13cb5ed50c78a4df404dc9"
server_key_iv: "5329e1203e2b4f40b89cdfa1267d8459"
server_key_version: "encryptionkey@0"
I20260812 01:23:33.071347 29109 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786497534195564-24578-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786497534195564-24578-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786497534195564-24578-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:23:33.081347 29109 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:23:33.083307 29109 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:23:33.083484 29109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:23:33.087829 29109 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.190:36549
I20260812 01:23:33.087869 29215 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.190:36549 every 8 connection(s)
I20260812 01:23:33.088337 29109 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
I20260812 01:23:33.089385 29222 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:23:33.090934 29222 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.0.190" port: 36549 } has no permanent_uuid. Determining permanent_uuid...
I20260812 01:23:33.093568 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 29109
I20260812 01:23:33.093698 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786497534195564-24578-0/minicluster-data/master-0/wal/instance
I20260812 01:23:33.093894 24578 external_mini_cluster.cc:1428] Setting key f6993df45539e174ff26528ef56a67e3
Aug 12 01:23:33 dist-test-slave-t3q3 krb5kdc[29087](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497813, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.190@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
I20260812 01:23:33.102205 29224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:33.091598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:36549 (local address 127.0.0.1:54670)
0812 01:23:33.091765 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0812 01:23:33.091801 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 01:23:33.092335 (+ 534us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:23:33.092623 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:23:33.092626 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:23:33.092880 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:23:33.093061 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:23:33.093065 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:23:33.094549 (+ 1484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:23:33.094550 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:23:33.095227 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:23:33.095229 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:23:33.095964 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:23:33.095966 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:23:33.096049 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:33.096678 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:23:33.096690 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:23:33.098381 (+ 1691us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:23:33.100574 (+ 2193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:23:33.100582 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:23:33.100584 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:23:33.100789 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:23:33.101106 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:23:33.101107 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:23:33.101107 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:23:33.101198 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:23:33.101576 (+ 378us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:23:33.101582 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:23:33.101647 (+ 65us) client_negotiation.cc:769] Sending connection context
0812 01:23:33.101764 (+ 117us) client_negotiation.cc:240] Negotiation successful
0812 01:23:33.102021 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"mutex_wait_us":105,"thread_start_us":77,"threads_started":1}
I20260812 01:23:33.102207 29225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:33.091759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54670 (local address 127.24.0.190:36549)
0812 01:23:33.091894 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:33.091899 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:33.091939 (+ 40us) server_negotiation.cc:407] Connection header received
0812 01:23:33.092483 (+ 544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:33.092485 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:33.092503 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:33.092587 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:23:33.093160 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:33.094449 (+ 1289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:33.095322 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:33.095835 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:33.095930 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:33.098502 (+ 2572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:23:33.098510 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:23:33.098510 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:23:33.098518 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:23:33.100462 (+ 1944us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:23:33.100898 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:23:33.100901 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:23:33.100901 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:23:33.101016 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:23:33.101275 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:23:33.101276 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:23:33.101277 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:23:33.101495 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:23:33.101597 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:33.101799 (+ 202us) server_negotiation.cc:299] Negotiation successful
0812 01:23:33.101949 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":77,"threads_started":1}
WARNING: no policy specified for kudu/127.24.0.189@KRBTEST.COM; defaulting to no policy
I20260812 01:23:33.103051 29222 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.0.189" port: 42977 } has no permanent_uuid. Determining permanent_uuid...
I20260812 01:23:33.103441 29224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:33.103312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.189:42977 (local address 127.0.0.1:57892)
0812 01:23:33.103374 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0812 01:23:33.103420 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.189:42977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260812 01:23:33.103611 29134 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.0.189:42977: connect: Connection refused (error 111)
I20260812 01:23:33.104126 29224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:33.103991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.189:42977 (local address 127.0.0.1:57900)
0812 01:23:33.104050 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0812 01:23:33.104102 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.189:42977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260812 01:23:33.104311 29222 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.0.189:42977: Network error: Client connection negotiation failed: client connection to 127.24.0.189:42977: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.0.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.0.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.189.keytab.
Entry for principal kudu/127.24.0.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.189.keytab.
I20260812 01:23:33.122422 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786497534195564-24578-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786497534195564-24578-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786497534195564-24578-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786497534195564-24578-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.189:42977
--webserver_interface=127.24.0.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.189.keytab
--principal=kudu/127.24.0.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:46113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.0.190:36549,127.24.0.189:42977,127.24.0.188:33185
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 01:23:33.153585 29222 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.0.189" port: 42977 } attempt: 1
I20260812 01:23:33.154139 29224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:33.153977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.189:42977 (local address 127.0.0.1:57912)
0812 01:23:33.154050 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0812 01:23:33.154102 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.189:42977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260812 01:23:33.154543 29224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:33.154428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.189:42977 (local address 127.0.0.1:57926)
0812 01:23:33.154476 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0812 01:23:33.154505 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.189:42977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260812 01:23:33.154646 29222 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.0.189:42977: Network error: Client connection negotiation failed: client connection to 127.24.0.189:42977: connect: Connection refused (error 111)
W20260812 01:23:33.206069 29231 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 01:23:33.206278 29231 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:23:33.206297 29231 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:23:33.206317 29231 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:23:33.207360 29231 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:23:33.207394 29231 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:23:33.207408 29231 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:23:33.207422 29231 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:23:33.207433 29231 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:23:33.209049 29231 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:46113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786497534195564-24578-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786497534195564-24578-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.24.0.190:36549,127.24.0.189:42977,127.24.0.188:33185
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.189.keytab
--principal=kudu/127.24.0.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.189:42977
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786497534195564-24578-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.24.0.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29231
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786497534195564-24578-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:23:33.209301 29231 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:23:33.209546 29231 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:23:33.214529 29242 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:23:33.214534 29238 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:23:33.214563 29244 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:23:33.214919 29231 server_base.cc:1034] running on GCE node
I20260812 01:23:33.215220 29231 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:23:33.215566 29231 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:23:33.216734 29231 hybrid_clock.cc:648] HybridClock initialized: now 1786497813216713 us; error 41 us; skew 500 ppm
Aug 12 01:23:33 dist-test-slave-t3q3 krb5kdc[29087](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497813, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:23:33.219431 29231 init.cc:377] Logged in from keytab as kudu/127.24.0.189@KRBTEST.COM (short username kudu)
I20260812 01:23:33.219949 29231 webserver.cc:460] Webserver started at http://127.24.0.189:40541/ using document root <none> and password file <none>
I20260812 01:23:33.220263 29231 fs_manager.cc:359] Metadata directory not provided
I20260812 01:23:33.220340 29231 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:23:33.220495 29231 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:23:33.221582 29231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786497534195564-24578-0/minicluster-data/master-1/data/instance:
uuid: "0e451071ddd44da1a0e8a17e726f5df0"
format_stamp: "Formatted at 2026-08-12 01:23:33 on dist-test-slave-t3q3"
server_key: "cc2fcabf099e442681a18c83b3cb1e1d"
server_key_iv: "cbb5b678441ba8e15217f7206374086a"
server_key_version: "encryptionkey@0"
I20260812 01:23:33.222002 29231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786497534195564-24578-0/minicluster-data/master-1/wal/instance:
uuid: "0e451071ddd44da1a0e8a17e726f5df0"
format_stamp: "Formatted at 2026-08-12 01:23:33 on dist-test-slave-t3q3"
server_key: "cc2fcabf099e442681a18c83b3cb1e1d"
server_key_iv: "cbb5b678441ba8e15217f7206374086a"
server_key_version: "encryptionkey@0"
I20260812 01:23:33.222770 29222 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.0.189" port: 42977 } attempt: 2
I20260812 01:23:33.223170 29224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:33.223025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.189:42977 (local address 127.0.0.1:57938)
0812 01:23:33.223091 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0812 01:23:33.223127 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.189:42977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260812 01:23:33.223527 29224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:33.223432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.189:42977 (local address 127.0.0.1:57952)
0812 01:23:33.223483 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0812 01:23:33.223507 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.189:42977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260812 01:23:33.223623 29222 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.0.189:42977: Network error: Client connection negotiation failed: client connection to 127.24.0.189:42977: connect: Connection refused (error 111)
I20260812 01:23:33.223608 29231 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 01:23:33.224478 29256 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:23:33.224687 29231 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 01:23:33.224788 29231 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786497534195564-24578-0/minicluster-data/master-1/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786497534195564-24578-0/minicluster-data/master-1/wal
uuid: "0e451071ddd44da1a0e8a17e726f5df0"
format_stamp: "Formatted at 2026-08-12 01:23:33 on dist-test-slave-t3q3"
server_key: "cc2fcabf099e442681a18c83b3cb1e1d"
server_key_iv: "cbb5b678441ba8e15217f7206374086a"
server_key_version: "encryptionkey@0"
I20260812 01:23:33.224864 29231 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786497534195564-24578-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786497534195564-24578-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786497534195564-24578-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:23:33.232831 29231 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:23:33.235188 29231 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:23:33.235354 29231 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:23:33.239578 29231 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.189:42977
I20260812 01:23:33.239634 29344 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.189:42977 every 8 connection(s)
I20260812 01:23:33.239989 29231 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786497534195564-24578-0/minicluster-data/master-1/data/info.pb
I20260812 01:23:33.240989 29345 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:23:33.242506 29345 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.0.190" port: 36549 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 01:23:33 dist-test-slave-t3q3 krb5kdc[29087](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497813, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.189@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
I20260812 01:23:33.248149 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 29231
I20260812 01:23:33.248273 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786497534195564-24578-0/minicluster-data/master-1/wal/instance
I20260812 01:23:33.248473 24578 external_mini_cluster.cc:1428] Setting key e605e09523b46e0cab8ba6a999e13437
I20260812 01:23:33.251407 29225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:33.243209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54678 (local address 127.24.0.190:36549)
0812 01:23:33.243292 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:33.243295 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:33.243466 (+ 171us) server_negotiation.cc:407] Connection header received
0812 01:23:33.243884 (+ 418us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:33.243885 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:33.243898 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:33.243954 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:23:33.244622 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:33.245082 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:33.246019 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:33.246213 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:33.246265 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:33.248349 (+ 2084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:23:33.248351 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:23:33.248351 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:23:33.248359 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:23:33.249956 (+ 1597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:23:33.250397 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:23:33.250399 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:23:33.250400 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:23:33.250465 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:23:33.250750 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:23:33.250751 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:23:33.250751 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:23:33.250871 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:23:33.250954 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:33.251230 (+ 276us) server_negotiation.cc:299] Negotiation successful
0812 01:23:33.251317 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 01:23:33.251585 29350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:33.243150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:36549 (local address 127.0.0.1:54678)
0812 01:23:33.243344 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0812 01:23:33.243379 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0812 01:23:33.243802 (+ 423us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:23:33.244016 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:23:33.244019 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:23:33.244352 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:23:33.244536 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:23:33.244541 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:23:33.245179 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:23:33.245180 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:23:33.245939 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:23:33.245941 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:23:33.246303 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:23:33.246304 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:23:33.246406 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:33.246828 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:23:33.246836 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:23:33.248265 (+ 1429us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:23:33.250056 (+ 1791us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:23:33.250063 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:23:33.250064 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:23:33.250284 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:23:33.250546 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:23:33.250547 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:23:33.250547 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:23:33.250665 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:23:33.250945 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:23:33.250953 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:23:33.251091 (+ 138us) client_negotiation.cc:769] Sending connection context
0812 01:23:33.251208 (+ 117us) client_negotiation.cc:240] Negotiation successful
0812 01:23:33.251358 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":112,"threads_started":1}
I20260812 01:23:33.252290 29345 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.0.189" port: 42977 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 01:23:33 dist-test-slave-t3q3 krb5kdc[29087](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497813, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.189@KRBTEST.COM for kudu/127.24.0.189@KRBTEST.COM
WARNING: no policy specified for kudu/127.24.0.188@KRBTEST.COM; defaulting to no policy
I20260812 01:23:33.260540 29350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:33.252555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.189:42977 (local address 127.0.0.1:57964)
0812 01:23:33.252612 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0812 01:23:33.252623 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 01:23:33.252666 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:23:33.252948 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:23:33.252948 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:23:33.253078 (+ 130us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:23:33.253124 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:23:33.253125 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:23:33.254497 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:23:33.254499 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:23:33.255386 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:23:33.255389 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:23:33.255856 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:23:33.255857 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:23:33.255937 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:33.256333 (+ 396us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:23:33.256338 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:23:33.257720 (+ 1382us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:23:33.259546 (+ 1826us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:23:33.259547 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:23:33.259548 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:23:33.259739 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:23:33.259940 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:23:33.259941 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:23:33.259941 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:23:33.259999 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:23:33.260309 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:23:33.260310 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:23:33.260345 (+ 35us) client_negotiation.cc:769] Sending connection context
0812 01:23:33.260401 (+ 56us) client_negotiation.cc:240] Negotiation successful
0812 01:23:33.260478 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260812 01:23:33.260700 29355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:33.252621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57964 (local address 127.24.0.189:42977)
0812 01:23:33.252784 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:33.252789 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:33.252812 (+ 23us) server_negotiation.cc:407] Connection header received
0812 01:23:33.252835 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:33.252837 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:33.252851 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:33.252930 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:23:33.253208 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:33.254387 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:33.255477 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:33.255773 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:33.255828 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:33.257823 (+ 1995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:23:33.257833 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:23:33.257833 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:23:33.257841 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:23:33.259432 (+ 1591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:23:33.259817 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:23:33.259820 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:23:33.259820 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:23:33.259877 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:23:33.260055 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:23:33.260056 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:23:33.260056 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:23:33.260243 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:23:33.260309 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:33.260436 (+ 127us) server_negotiation.cc:299] Negotiation successful
0812 01:23:33.260530 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":28,"server-negotiator.queue_time_us":131,"thread_start_us":85,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.0.188@KRBTEST.COM" created.
I20260812 01:23:33.261310 29345 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.0.188" port: 33185 } has no permanent_uuid. Determining permanent_uuid...
I20260812 01:23:33.261641 29350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:33.261552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.188:33185 (local address 127.0.0.1:39044)
0812 01:23:33.261583 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0812 01:23:33.261625 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.188:33185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
W20260812 01:23:33.261772 29263 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.0.188:33185: connect: Connection refused (error 111)
I20260812 01:23:33.262230 29350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:33.262117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.188:33185 (local address 127.0.0.1:39050)
0812 01:23:33.262178 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0812 01:23:33.262203 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.188:33185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260812 01:23:33.262365 29345 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.0.188:33185: Network error: Client connection negotiation failed: client connection to 127.24.0.188:33185: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.0.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.188.keytab.
Entry for principal kudu/127.24.0.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.188.keytab.
I20260812 01:23:33.276734 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786497534195564-24578-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786497534195564-24578-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786497534195564-24578-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786497534195564-24578-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.188:33185
--webserver_interface=127.24.0.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.188.keytab
--principal=kudu/127.24.0.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:46113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.0.190:36549,127.24.0.189:42977,127.24.0.188:33185
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 01:23:33.311637 29345 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.0.188" port: 33185 } attempt: 1
I20260812 01:23:33.312245 29350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:33.312044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.188:33185 (local address 127.0.0.1:39052)
0812 01:23:33.312143 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0812 01:23:33.312206 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.188:33185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260812 01:23:33.312651 29350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:33.312544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.188:33185 (local address 127.0.0.1:39060)
0812 01:23:33.312599 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0812 01:23:33.312630 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.188:33185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260812 01:23:33.312762 29345 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.0.188:33185: Network error: Client connection negotiation failed: client connection to 127.24.0.188:33185: connect: Connection refused (error 111)
I20260812 01:23:33.314785 29222 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.0.189" port: 42977 } attempt: 3
Aug 12 01:23:33 dist-test-slave-t3q3 krb5kdc[29087](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497813, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.190@KRBTEST.COM for kudu/127.24.0.189@KRBTEST.COM
I20260812 01:23:33.321982 29224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:33.315084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.189:42977 (local address 127.0.0.1:57978)
0812 01:23:33.315149 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0812 01:23:33.315160 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 01:23:33.315230 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:23:33.315380 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:23:33.315381 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:23:33.315559 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:23:33.315625 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:23:33.315627 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:23:33.316383 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:23:33.316384 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:23:33.316917 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:23:33.316919 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:23:33.317337 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:23:33.317338 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:23:33.317421 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:33.317912 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:23:33.317917 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:23:33.319300 (+ 1383us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:23:33.321065 (+ 1765us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:23:33.321065 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:23:33.321066 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:23:33.321230 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:23:33.321438 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:23:33.321439 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:23:33.321439 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:23:33.321494 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:23:33.321752 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:23:33.321753 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:23:33.321792 (+ 39us) client_negotiation.cc:769] Sending connection context
0812 01:23:33.321844 (+ 52us) client_negotiation.cc:240] Negotiation successful
0812 01:23:33.321918 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260812 01:23:33.322079 29355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:33.315159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57978 (local address 127.24.0.189:42977)
0812 01:23:33.315227 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:33.315230 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:33.315242 (+ 12us) server_negotiation.cc:407] Connection header received
0812 01:23:33.315292 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:33.315293 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:33.315306 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:33.315375 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:23:33.315728 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:33.316296 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:33.317022 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:33.317247 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:33.317326 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:33.319394 (+ 2068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:23:33.319396 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:23:33.319396 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:23:33.319404 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:23:33.320983 (+ 1579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:23:33.321306 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:23:33.321307 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:23:33.321308 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:23:33.321369 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:23:33.321558 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:23:33.321559 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:23:33.321560 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:23:33.321687 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:23:33.321761 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:33.321904 (+ 143us) server_negotiation.cc:299] Negotiation successful
0812 01:23:33.321998 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 01:23:33.322469 29222 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.0.188" port: 33185 } has no permanent_uuid. Determining permanent_uuid...
I20260812 01:23:33.322779 29224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:33.322696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.188:33185 (local address 127.0.0.1:39070)
0812 01:23:33.322728 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0812 01:23:33.322758 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.188:33185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260812 01:23:33.323082 29224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:33.323027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.188:33185 (local address 127.0.0.1:39086)
0812 01:23:33.323049 (+ 22us) negotiation.cc:107] Waiting for socket to connect
0812 01:23:33.323068 (+ 19us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.188:33185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":13}
W20260812 01:23:33.323201 29222 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.0.188:33185: Network error: Client connection negotiation failed: client connection to 127.24.0.188:33185: connect: Connection refused (error 111)
I20260812 01:23:33.354388 29222 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.0.188" port: 33185 } attempt: 1
I20260812 01:23:33.354915 29224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:33.354753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.188:33185 (local address 127.0.0.1:39094)
0812 01:23:33.354834 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0812 01:23:33.354880 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.188:33185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260812 01:23:33.355304 29224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:33.355217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.188:33185 (local address 127.0.0.1:39110)
0812 01:23:33.355248 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0812 01:23:33.355277 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.188:33185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
W20260812 01:23:33.355408 29222 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.0.188:33185: Network error: Client connection negotiation failed: client connection to 127.24.0.188:33185: connect: Connection refused (error 111)
W20260812 01:23:33.362372 29363 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 01:23:33.362561 29363 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:23:33.362578 29363 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:23:33.362597 29363 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:23:33.363623 29363 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:23:33.363664 29363 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:23:33.363678 29363 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:23:33.363691 29363 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:23:33.363703 29363 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:23:33.365159 29363 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:46113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786497534195564-24578-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786497534195564-24578-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.24.0.190:36549,127.24.0.189:42977,127.24.0.188:33185
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.188.keytab
--principal=kudu/127.24.0.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.188:33185
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786497534195564-24578-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.24.0.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29363
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786497534195564-24578-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:23:33.365348 29363 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:23:33.365515 29363 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:23:33.370188 29376 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:23:33.370203 29372 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:23:33.370196 29374 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:23:33.370651 29363 server_base.cc:1034] running on GCE node
I20260812 01:23:33.370939 29363 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:23:33.371254 29363 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:23:33.372428 29363 hybrid_clock.cc:648] HybridClock initialized: now 1786497813372402 us; error 38 us; skew 500 ppm
Aug 12 01:23:33 dist-test-slave-t3q3 krb5kdc[29087](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497813, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:23:33.378546 29363 init.cc:377] Logged in from keytab as kudu/127.24.0.188@KRBTEST.COM (short username kudu)
I20260812 01:23:33.379009 29363 webserver.cc:460] Webserver started at http://127.24.0.188:34627/ using document root <none> and password file <none>
I20260812 01:23:33.379253 29363 fs_manager.cc:359] Metadata directory not provided
I20260812 01:23:33.379328 29363 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:23:33.379550 29363 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:23:33.380728 29363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786497534195564-24578-0/minicluster-data/master-2/data/instance:
uuid: "e447e798d4d2499ebd1fe2c1ad7cbda4"
format_stamp: "Formatted at 2026-08-12 01:23:33 on dist-test-slave-t3q3"
server_key: "6bda5c2d6eb16c143317e8b2a578b7df"
server_key_iv: "d3650aca7aa356df3a3fc18ee27132d4"
server_key_version: "encryptionkey@0"
I20260812 01:23:33.380893 29345 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.0.188" port: 33185 } attempt: 2
I20260812 01:23:33.381150 29363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786497534195564-24578-0/minicluster-data/master-2/wal/instance:
uuid: "e447e798d4d2499ebd1fe2c1ad7cbda4"
format_stamp: "Formatted at 2026-08-12 01:23:33 on dist-test-slave-t3q3"
server_key: "6bda5c2d6eb16c143317e8b2a578b7df"
server_key_iv: "d3650aca7aa356df3a3fc18ee27132d4"
server_key_version: "encryptionkey@0"
I20260812 01:23:33.381320 29350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:33.381187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.188:33185 (local address 127.0.0.1:39112)
0812 01:23:33.381250 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0812 01:23:33.381282 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.188:33185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260812 01:23:33.381680 29350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:33.381577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.188:33185 (local address 127.0.0.1:39126)
0812 01:23:33.381628 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0812 01:23:33.381659 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.188:33185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260812 01:23:33.381767 29345 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.0.188:33185: Network error: Client connection negotiation failed: client connection to 127.24.0.188:33185: connect: Connection refused (error 111)
I20260812 01:23:33.382758 29363 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 01:23:33.383514 29384 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:23:33.383710 29363 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 01:23:33.383816 29363 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786497534195564-24578-0/minicluster-data/master-2/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786497534195564-24578-0/minicluster-data/master-2/wal
uuid: "e447e798d4d2499ebd1fe2c1ad7cbda4"
format_stamp: "Formatted at 2026-08-12 01:23:33 on dist-test-slave-t3q3"
server_key: "6bda5c2d6eb16c143317e8b2a578b7df"
server_key_iv: "d3650aca7aa356df3a3fc18ee27132d4"
server_key_version: "encryptionkey@0"
I20260812 01:23:33.383908 29363 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786497534195564-24578-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786497534195564-24578-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786497534195564-24578-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:23:33.396245 29363 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:23:33.398027 29363 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:23:33.398200 29363 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:23:33.402621 29363 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.188:33185
I20260812 01:23:33.402649 29483 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.188:33185 every 8 connection(s)
I20260812 01:23:33.403031 29363 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786497534195564-24578-0/minicluster-data/master-2/data/info.pb
I20260812 01:23:33.404049 29484 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:23:33.405689 29484 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.0.190" port: 36549 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 01:23:33 dist-test-slave-t3q3 krb5kdc[29087](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497813, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.188@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
I20260812 01:23:33.412235 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 29363
I20260812 01:23:33.412357 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786497534195564-24578-0/minicluster-data/master-2/wal/instance
I20260812 01:23:33.412542 24578 external_mini_cluster.cc:1428] Setting key 41f07607449b463e193dc2988f529df5
I20260812 01:23:33.414934 29225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:33.406415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54686 (local address 127.24.0.190:36549)
0812 01:23:33.406492 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:33.406494 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:33.406701 (+ 207us) server_negotiation.cc:407] Connection header received
0812 01:23:33.407122 (+ 421us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:33.407123 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:33.407133 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:33.407197 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:23:33.407837 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:33.408380 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:33.409326 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:33.409504 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:33.409556 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:33.411603 (+ 2047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:23:33.411605 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:23:33.411605 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:23:33.411611 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:23:33.413432 (+ 1821us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:23:33.413927 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:23:33.413928 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:23:33.413928 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:23:33.413979 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:23:33.414282 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:23:33.414283 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:23:33.414283 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:23:33.414416 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:23:33.414491 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:33.414757 (+ 266us) server_negotiation.cc:299] Negotiation successful
0812 01:23:33.414840 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 01:23:33.414952 29486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:33.406371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:36549 (local address 127.0.0.1:54686)
0812 01:23:33.406563 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0812 01:23:33.406604 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 01:23:33.407039 (+ 435us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:23:33.407271 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:23:33.407274 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:23:33.407559 (+ 285us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:23:33.407755 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:23:33.407759 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:23:33.408463 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:23:33.408464 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:23:33.409249 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:23:33.409251 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:23:33.409589 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:23:33.409590 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:23:33.409696 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:33.410171 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:23:33.410179 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:23:33.411512 (+ 1333us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:23:33.413513 (+ 2001us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:23:33.413521 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:23:33.413523 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:23:33.413843 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:23:33.414054 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:23:33.414055 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:23:33.414055 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:23:33.414211 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:23:33.414489 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:23:33.414494 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:23:33.414604 (+ 110us) client_negotiation.cc:769] Sending connection context
0812 01:23:33.414690 (+ 86us) client_negotiation.cc:240] Negotiation successful
0812 01:23:33.414801 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":117,"threads_started":1}
I20260812 01:23:33.415644 29484 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.0.189" port: 42977 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 01:23:33 dist-test-slave-t3q3 krb5kdc[29087](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497812, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.0.188@KRBTEST.COM
Aug 12 01:23:33 dist-test-slave-t3q3 krb5kdc[29087](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497813, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.188@KRBTEST.COM for kudu/127.24.0.189@KRBTEST.COM
I20260812 01:23:33.422499 29489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:33.413003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39138 (local address 127.24.0.188:33185)
0812 01:23:33.413212 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:33.413218 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:33.413240 (+ 22us) server_negotiation.cc:407] Connection header received
0812 01:23:33.413265 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:33.413267 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:33.413280 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:33.413369 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:23:33.414202 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:33.415224 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:33.416299 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:33.416602 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:33.416707 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:33.419283 (+ 2576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:23:33.419290 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:23:33.419290 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:23:33.419295 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:23:33.421242 (+ 1947us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:23:33.421633 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:23:33.421636 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:23:33.421636 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:23:33.421693 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:23:33.421915 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:23:33.421916 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:23:33.421916 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:23:33.422081 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:23:33.422253 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:33.422309 (+ 56us) server_negotiation.cc:299] Negotiation successful
0812 01:23:33.422391 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":179,"thread_start_us":91,"threads_started":1}
I20260812 01:23:33.422683 29486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:33.415918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.189:42977 (local address 127.0.0.1:57994)
0812 01:23:33.415971 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0812 01:23:33.415982 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 01:23:33.416019 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:23:33.416193 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:23:33.416194 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:23:33.416328 (+ 134us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:23:33.416370 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:23:33.416371 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:23:33.417015 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:23:33.417016 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:23:33.417845 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:23:33.417849 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:23:33.418253 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:23:33.418254 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:23:33.418335 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:33.418739 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:23:33.418743 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:23:33.419896 (+ 1153us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:23:33.421564 (+ 1668us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:23:33.421565 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:23:33.421566 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:23:33.421773 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:23:33.422007 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:23:33.422007 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:23:33.422008 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:23:33.422059 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:23:33.422353 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:23:33.422354 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:23:33.422397 (+ 43us) client_negotiation.cc:769] Sending connection context
0812 01:23:33.422473 (+ 76us) client_negotiation.cc:240] Negotiation successful
0812 01:23:33.422567 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260812 01:23:33.422830 29355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:33.416005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57994 (local address 127.24.0.189:42977)
0812 01:23:33.416076 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:33.416078 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:33.416087 (+ 9us) server_negotiation.cc:407] Connection header received
0812 01:23:33.416111 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:33.416112 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:33.416121 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:33.416171 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:23:33.416464 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:33.416911 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:33.417940 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:33.418136 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:33.418250 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:33.419989 (+ 1739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:23:33.419991 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:23:33.419991 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:23:33.419999 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:23:33.421484 (+ 1485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:23:33.421853 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:23:33.421853 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:23:33.421854 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:23:33.421934 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:23:33.422129 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:23:33.422130 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:23:33.422131 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:23:33.422272 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:23:33.422341 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:33.422523 (+ 182us) server_negotiation.cc:299] Negotiation successful
0812 01:23:33.422636 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 01:23:33.423202 29484 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.0.188" port: 33185 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 01:23:33 dist-test-slave-t3q3 krb5kdc[29087](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497813, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.188@KRBTEST.COM for kudu/127.24.0.188@KRBTEST.COM
Aug 12 01:23:33 dist-test-slave-t3q3 krb5kdc[29087](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497812, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
I20260812 01:23:33.430424 29486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:33.423462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.188:33185 (local address 127.0.0.1:39154)
0812 01:23:33.423500 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0812 01:23:33.423510 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 01:23:33.423544 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:23:33.423664 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:23:33.423664 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:23:33.423815 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:23:33.423851 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:23:33.423853 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:23:33.424733 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:23:33.424734 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:23:33.425398 (+ 664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:23:33.425400 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:23:33.425780 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:23:33.425781 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:23:33.425817 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:33.426212 (+ 395us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:23:33.426216 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:23:33.427531 (+ 1315us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:23:33.429341 (+ 1810us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:23:33.429342 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:23:33.429343 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:23:33.429514 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:23:33.429737 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:23:33.429737 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:23:33.429737 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:23:33.429789 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:23:33.430084 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:23:33.430085 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:23:33.430129 (+ 44us) client_negotiation.cc:769] Sending connection context
0812 01:23:33.430218 (+ 89us) client_negotiation.cc:240] Negotiation successful
0812 01:23:33.430310 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20}
I20260812 01:23:33.430586 29489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:33.423540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39154 (local address 127.24.0.188:33185)
0812 01:23:33.423576 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:33.423578 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:33.423584 (+ 6us) server_negotiation.cc:407] Connection header received
0812 01:23:33.423601 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:33.423602 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:33.423612 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:33.423661 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:23:33.423928 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:33.424639 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:33.425481 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:33.425663 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:33.425760 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:33.427603 (+ 1843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:23:33.427604 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:23:33.427605 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:23:33.427611 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:23:33.429255 (+ 1644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:23:33.429593 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:23:33.429594 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:23:33.429594 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:23:33.429653 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:23:33.429858 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:23:33.429859 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:23:33.429860 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:23:33.429992 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:23:33.430065 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:33.430239 (+ 174us) server_negotiation.cc:299] Negotiation successful
0812 01:23:33.430351 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":53,"server-negotiator.queue_time_us":20}
I20260812 01:23:33.430630 29222 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.0.188" port: 33185 } attempt: 2
I20260812 01:23:33.431983 29484 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e447e798d4d2499ebd1fe2c1ad7cbda4: Bootstrap starting.
I20260812 01:23:33.432871 29484 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e447e798d4d2499ebd1fe2c1ad7cbda4: Neither blocks nor log segments found. Creating new log.
I20260812 01:23:33.433204 29484 log.cc:824] T 00000000000000000000000000000000 P e447e798d4d2499ebd1fe2c1ad7cbda4: Log is configured to *not* fsync() on all Append() calls
I20260812 01:23:33.433800 29225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:33.423476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54702 (local address 127.24.0.190:36549)
0812 01:23:33.423518 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:33.423520 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:33.423528 (+ 8us) server_negotiation.cc:407] Connection header received
0812 01:23:33.423548 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:33.423549 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:33.423558 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:33.423622 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:23:33.424253 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:33.424880 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:33.425736 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:33.425954 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:33.426003 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:33.428456 (+ 2453us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:23:33.428457 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:23:33.428457 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:23:33.428463 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:23:33.430024 (+ 1561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:23:33.430417 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:23:33.430418 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:23:33.430418 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:23:33.430474 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:23:33.430716 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:23:33.430718 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:23:33.430718 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:23:33.430842 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:23:33.431035 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:33.433592 (+ 2557us) server_negotiation.cc:299] Negotiation successful
0812 01:23:33.433694 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 01:23:33.434051 29484 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e447e798d4d2499ebd1fe2c1ad7cbda4: No bootstrap required, opened a new log
Aug 12 01:23:33 dist-test-slave-t3q3 krb5kdc[29087](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497813, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.190@KRBTEST.COM for kudu/127.24.0.188@KRBTEST.COM
I20260812 01:23:33.436187 29484 raft_consensus.cc:348] T 00000000000000000000000000000000 P e447e798d4d2499ebd1fe2c1ad7cbda4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83c4efc95926420d8a7bf318f9dafd8f" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 36549 } } peers { permanent_uuid: "0e451071ddd44da1a0e8a17e726f5df0" member_type: VOTER last_known_addr { host: "127.24.0.189" port: 42977 } } peers { permanent_uuid: "e447e798d4d2499ebd1fe2c1ad7cbda4" member_type: VOTER last_known_addr { host: "127.24.0.188" port: 33185 } }
I20260812 01:23:33.436350 29484 raft_consensus.cc:729] T 00000000000000000000000000000000 P e447e798d4d2499ebd1fe2c1ad7cbda4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e447e798d4d2499ebd1fe2c1ad7cbda4, State: Initialized, Role: FOLLOWER
I20260812 01:23:33.436445 29484 consensus_queue.cc:260] T 00000000000000000000000000000000 P e447e798d4d2499ebd1fe2c1ad7cbda4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83c4efc95926420d8a7bf318f9dafd8f" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 36549 } } peers { permanent_uuid: "0e451071ddd44da1a0e8a17e726f5df0" member_type: VOTER last_known_addr { host: "127.24.0.189" port: 42977 } } peers { permanent_uuid: "e447e798d4d2499ebd1fe2c1ad7cbda4" member_type: VOTER last_known_addr { host: "127.24.0.188" port: 33185 } }
I20260812 01:23:33.436813 29484 sys_catalog.cc:564] T 00000000000000000000000000000000 P e447e798d4d2499ebd1fe2c1ad7cbda4 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:23:33.436757 29492 sys_catalog.cc:455] T 00000000000000000000000000000000 P e447e798d4d2499ebd1fe2c1ad7cbda4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83c4efc95926420d8a7bf318f9dafd8f" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 36549 } } peers { permanent_uuid: "0e451071ddd44da1a0e8a17e726f5df0" member_type: VOTER last_known_addr { host: "127.24.0.189" port: 42977 } } peers { permanent_uuid: "e447e798d4d2499ebd1fe2c1ad7cbda4" member_type: VOTER last_known_addr { host: "127.24.0.188" port: 33185 } } }
I20260812 01:23:33.436879 29492 sys_catalog.cc:458] T 00000000000000000000000000000000 P e447e798d4d2499ebd1fe2c1ad7cbda4 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 01:23:33.437848 29489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:33.430934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39168 (local address 127.24.0.188:33185)
0812 01:23:33.430965 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:33.430967 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:33.431128 (+ 161us) server_negotiation.cc:407] Connection header received
0812 01:23:33.431197 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:33.431198 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:33.431207 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:33.431262 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:23:33.431523 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:33.431947 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:33.432695 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:33.432892 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:33.433006 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:33.434863 (+ 1857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:23:33.434865 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:23:33.434865 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:23:33.434874 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:23:33.436462 (+ 1588us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:23:33.436904 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:23:33.436905 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:23:33.436906 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:23:33.436965 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:23:33.437221 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:23:33.437223 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:23:33.437223 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:23:33.437368 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:23:33.437466 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:33.437646 (+ 180us) server_negotiation.cc:299] Negotiation successful
0812 01:23:33.437734 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260812 01:23:33.437829 29224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:33.430883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.188:33185 (local address 127.0.0.1:39168)
0812 01:23:33.431052 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0812 01:23:33.431062 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 01:23:33.431139 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:23:33.431266 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:23:33.431267 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:23:33.431405 (+ 138us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:23:33.431436 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:23:33.431437 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:23:33.432031 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:23:33.432032 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:23:33.432584 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:23:33.432585 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:23:33.433008 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:23:33.433009 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:23:33.433067 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:33.433493 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:23:33.433499 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:23:33.434777 (+ 1278us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:23:33.436545 (+ 1768us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:23:33.436546 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:23:33.436546 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:23:33.436780 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:23:33.437067 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:23:33.437068 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:23:33.437069 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:23:33.437137 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:23:33.437476 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:23:33.437477 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:23:33.437531 (+ 54us) client_negotiation.cc:769] Sending connection context
0812 01:23:33.437618 (+ 87us) client_negotiation.cc:240] Negotiation successful
0812 01:23:33.437736 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152}
I20260812 01:23:33.439587 29222 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83c4efc95926420d8a7bf318f9dafd8f: Bootstrap starting.
I20260812 01:23:33.440325 29222 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 83c4efc95926420d8a7bf318f9dafd8f: Neither blocks nor log segments found. Creating new log.
I20260812 01:23:33.440616 29222 log.cc:824] T 00000000000000000000000000000000 P 83c4efc95926420d8a7bf318f9dafd8f: Log is configured to *not* fsync() on all Append() calls
I20260812 01:23:33.441471 29222 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83c4efc95926420d8a7bf318f9dafd8f: No bootstrap required, opened a new log
Aug 12 01:23:33 dist-test-slave-t3q3 krb5kdc[29087](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497812, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.0.189@KRBTEST.COM
I20260812 01:23:33.443238 29222 raft_consensus.cc:348] T 00000000000000000000000000000000 P 83c4efc95926420d8a7bf318f9dafd8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83c4efc95926420d8a7bf318f9dafd8f" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 36549 } } peers { permanent_uuid: "0e451071ddd44da1a0e8a17e726f5df0" member_type: VOTER last_known_addr { host: "127.24.0.189" port: 42977 } } peers { permanent_uuid: "e447e798d4d2499ebd1fe2c1ad7cbda4" member_type: VOTER last_known_addr { host: "127.24.0.188" port: 33185 } }
I20260812 01:23:33.443382 29222 raft_consensus.cc:729] T 00000000000000000000000000000000 P 83c4efc95926420d8a7bf318f9dafd8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83c4efc95926420d8a7bf318f9dafd8f, State: Initialized, Role: FOLLOWER
I20260812 01:23:33.443465 29222 consensus_queue.cc:260] T 00000000000000000000000000000000 P 83c4efc95926420d8a7bf318f9dafd8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83c4efc95926420d8a7bf318f9dafd8f" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 36549 } } peers { permanent_uuid: "0e451071ddd44da1a0e8a17e726f5df0" member_type: VOTER last_known_addr { host: "127.24.0.189" port: 42977 } } peers { permanent_uuid: "e447e798d4d2499ebd1fe2c1ad7cbda4" member_type: VOTER last_known_addr { host: "127.24.0.188" port: 33185 } }
I20260812 01:23:33.443833 29222 sys_catalog.cc:564] T 00000000000000000000000000000000 P 83c4efc95926420d8a7bf318f9dafd8f [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:23:33.443876 29511 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83c4efc95926420d8a7bf318f9dafd8f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83c4efc95926420d8a7bf318f9dafd8f" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 36549 } } peers { permanent_uuid: "0e451071ddd44da1a0e8a17e726f5df0" member_type: VOTER last_known_addr { host: "127.24.0.189" port: 42977 } } peers { permanent_uuid: "e447e798d4d2499ebd1fe2c1ad7cbda4" member_type: VOTER last_known_addr { host: "127.24.0.188" port: 33185 } } }
I20260812 01:23:33.443962 29511 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83c4efc95926420d8a7bf318f9dafd8f [sys.catalog]: This master's current role is: FOLLOWER
W20260812 01:23:33.444439 29507 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e447e798d4d2499ebd1fe2c1ad7cbda4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 01:23:33.444489 29507 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 01:23:33.441632 29363 master_runner.cc:186] Error getting master registration for 127.24.0.190:36549: OK, instance_id { permanent_uuid: "83c4efc95926420d8a7bf318f9dafd8f" instance_seqno: 1786497813086592 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 01:23:33.444617 29363 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260812 01:23:33.445727 29532 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 83c4efc95926420d8a7bf318f9dafd8f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 01:23:33.445851 29532 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 01:23:33.446326 29355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:33.437070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57998 (local address 127.24.0.189:42977)
0812 01:23:33.437109 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:33.437111 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:33.437118 (+ 7us) server_negotiation.cc:407] Connection header received
0812 01:23:33.437137 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:33.437138 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:33.437147 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:33.437226 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:23:33.438121 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:33.438887 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:33.439624 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:33.439895 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:33.439977 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:33.442639 (+ 2662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:23:33.442641 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:23:33.442641 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:23:33.442650 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:23:33.444622 (+ 1972us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:23:33.445045 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:23:33.445046 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:23:33.445046 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:23:33.445117 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:23:33.445391 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:23:33.445393 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:23:33.445393 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:23:33.445539 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:23:33.445721 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:33.446158 (+ 437us) server_negotiation.cc:299] Negotiation successful
0812 01:23:33.446239 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 01:23:33.447559 24578 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 01:23:33.448593 29180 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "83c4efc95926420d8a7bf318f9dafd8f"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54702
I20260812 01:23:33.448719 29180 raft_consensus.cc:482] T 00000000000000000000000000000000 P 83c4efc95926420d8a7bf318f9dafd8f [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 01:23:33.448763 29180 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 83c4efc95926420d8a7bf318f9dafd8f [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:23:33.446595 29109 master_runner.cc:186] Error getting master registration for 127.24.0.189:42977: OK, instance_id { permanent_uuid: "0e451071ddd44da1a0e8a17e726f5df0" instance_seqno: 1786497813238428 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 01:23:33.449636 29180 raft_consensus.cc:504] T 00000000000000000000000000000000 P 83c4efc95926420d8a7bf318f9dafd8f [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83c4efc95926420d8a7bf318f9dafd8f" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 36549 } } peers { permanent_uuid: "0e451071ddd44da1a0e8a17e726f5df0" member_type: VOTER last_known_addr { host: "127.24.0.189" port: 42977 } } peers { permanent_uuid: "e447e798d4d2499ebd1fe2c1ad7cbda4" member_type: VOTER last_known_addr { host: "127.24.0.188" port: 33185 } }
I20260812 01:23:33.449712 29109 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 01:23:33.449905 29180 leader_election.cc:288] T 00000000000000000000000000000000 P 83c4efc95926420d8a7bf318f9dafd8f [CANDIDATE]: Term 1 election: Requested vote from peers 0e451071ddd44da1a0e8a17e726f5df0 (127.24.0.189:42977), e447e798d4d2499ebd1fe2c1ad7cbda4 (127.24.0.188:33185)
I20260812 01:23:33.450254 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "83c4efc95926420d8a7bf318f9dafd8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "e447e798d4d2499ebd1fe2c1ad7cbda4"
I20260812 01:23:33.450379 29446 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e447e798d4d2499ebd1fe2c1ad7cbda4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:23:33.451207 29446 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e447e798d4d2499ebd1fe2c1ad7cbda4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83c4efc95926420d8a7bf318f9dafd8f in term 1.
I20260812 01:23:33.451440 29138 leader_election.cc:302] T 00000000000000000000000000000000 P 83c4efc95926420d8a7bf318f9dafd8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83c4efc95926420d8a7bf318f9dafd8f, e447e798d4d2499ebd1fe2c1ad7cbda4; no voters:
I20260812 01:23:33.451579 29511 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 83c4efc95926420d8a7bf318f9dafd8f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:23:33.451792 29511 raft_consensus.cc:686] T 00000000000000000000000000000000 P 83c4efc95926420d8a7bf318f9dafd8f [term 1 LEADER]: Becoming Leader. State: Replica: 83c4efc95926420d8a7bf318f9dafd8f, State: Running, Role: LEADER
I20260812 01:23:33.451905 29511 consensus_queue.cc:237] T 00000000000000000000000000000000 P 83c4efc95926420d8a7bf318f9dafd8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83c4efc95926420d8a7bf318f9dafd8f" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 36549 } } peers { permanent_uuid: "0e451071ddd44da1a0e8a17e726f5df0" member_type: VOTER last_known_addr { host: "127.24.0.189" port: 42977 } } peers { permanent_uuid: "e447e798d4d2499ebd1fe2c1ad7cbda4" member_type: VOTER last_known_addr { host: "127.24.0.188" port: 33185 } }
I20260812 01:23:33.452410 29511 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83c4efc95926420d8a7bf318f9dafd8f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 83c4efc95926420d8a7bf318f9dafd8f. Latest consensus state: current_term: 1 leader_uuid: "83c4efc95926420d8a7bf318f9dafd8f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83c4efc95926420d8a7bf318f9dafd8f" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 36549 } } peers { permanent_uuid: "0e451071ddd44da1a0e8a17e726f5df0" member_type: VOTER last_known_addr { host: "127.24.0.189" port: 42977 } } peers { permanent_uuid: "e447e798d4d2499ebd1fe2c1ad7cbda4" member_type: VOTER last_known_addr { host: "127.24.0.188" port: 33185 } } }
I20260812 01:23:33.452524 29511 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83c4efc95926420d8a7bf318f9dafd8f [sys.catalog]: This master's current role is: LEADER
I20260812 01:23:33.457068 29548 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:23:33.457294 29548 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:23:33.458981 29446 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e447e798d4d2499ebd1fe2c1ad7cbda4 [term 1 FOLLOWER]: Refusing update from remote peer 83c4efc95926420d8a7bf318f9dafd8f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:23:33.459425 29538 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 83c4efc95926420d8a7bf318f9dafd8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e447e798d4d2499ebd1fe2c1ad7cbda4" member_type: VOTER last_known_addr { host: "127.24.0.188" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:23:33.460861 29492 sys_catalog.cc:455] T 00000000000000000000000000000000 P e447e798d4d2499ebd1fe2c1ad7cbda4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 83c4efc95926420d8a7bf318f9dafd8f. Latest consensus state: current_term: 1 leader_uuid: "83c4efc95926420d8a7bf318f9dafd8f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83c4efc95926420d8a7bf318f9dafd8f" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 36549 } } peers { permanent_uuid: "0e451071ddd44da1a0e8a17e726f5df0" member_type: VOTER last_known_addr { host: "127.24.0.189" port: 42977 } } peers { permanent_uuid: "e447e798d4d2499ebd1fe2c1ad7cbda4" member_type: VOTER last_known_addr { host: "127.24.0.188" port: 33185 } } }
I20260812 01:23:33.460963 29492 sys_catalog.cc:458] T 00000000000000000000000000000000 P e447e798d4d2499ebd1fe2c1ad7cbda4 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 01:23:33.468787 29355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:33.456492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58000 (local address 127.24.0.189:42977)
0812 01:23:33.456538 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:33.456542 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:33.461121 (+ 4579us) server_negotiation.cc:407] Connection header received
0812 01:23:33.461241 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:33.461242 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:33.461254 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:33.461318 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:23:33.462135 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:33.462600 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:33.463650 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:33.463875 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:33.463958 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:33.465668 (+ 1710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:23:33.465670 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:23:33.465670 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:23:33.465678 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:23:33.467423 (+ 1745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:23:33.467801 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:23:33.467802 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:23:33.467803 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:23:33.467876 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:23:33.468097 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:23:33.468098 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:23:33.468098 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:23:33.468245 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:23:33.468343 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:33.468603 (+ 260us) server_negotiation.cc:299] Negotiation successful
0812 01:23:33.468692 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 01:23:33.470795 29538 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83c4efc95926420d8a7bf318f9dafd8f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "83c4efc95926420d8a7bf318f9dafd8f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83c4efc95926420d8a7bf318f9dafd8f" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 36549 } } peers { permanent_uuid: "0e451071ddd44da1a0e8a17e726f5df0" member_type: VOTER last_known_addr { host: "127.24.0.189" port: 42977 } } peers { permanent_uuid: "e447e798d4d2499ebd1fe2c1ad7cbda4" member_type: VOTER last_known_addr { host: "127.24.0.188" port: 33185 } } }
I20260812 01:23:33.470906 29538 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83c4efc95926420d8a7bf318f9dafd8f [sys.catalog]: This master's current role is: LEADER
I20260812 01:23:33.471503 29492 sys_catalog.cc:455] T 00000000000000000000000000000000 P e447e798d4d2499ebd1fe2c1ad7cbda4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "83c4efc95926420d8a7bf318f9dafd8f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83c4efc95926420d8a7bf318f9dafd8f" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 36549 } } peers { permanent_uuid: "0e451071ddd44da1a0e8a17e726f5df0" member_type: VOTER last_known_addr { host: "127.24.0.189" port: 42977 } } peers { permanent_uuid: "e447e798d4d2499ebd1fe2c1ad7cbda4" member_type: VOTER last_known_addr { host: "127.24.0.188" port: 33185 } } }
I20260812 01:23:33.471591 29492 sys_catalog.cc:458] T 00000000000000000000000000000000 P e447e798d4d2499ebd1fe2c1ad7cbda4 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 01:23:33.471575 29548 catalog_manager.cc:1347] Generated new cluster ID: f9a41c0981b74ed49ae759973845ff3b
I20260812 01:23:33.471688 29548 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 01:23:33.472882 29553 mvcc.cc:205] Tried to move back new op lower bound from 7317495043925270528 to 7317495043900182528. Current Snapshot: MvccSnapshot[applied={T|T < 7317495043925270528}]
I20260812 01:23:33.472882 29345 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.0.188" port: 33185 } attempt: 3
I20260812 01:23:33.476688 29489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:33.456957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39180 (local address 127.24.0.188:33185)
0812 01:23:33.456999 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:33.457001 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:33.468361 (+ 11360us) server_negotiation.cc:407] Connection header received
0812 01:23:33.469074 (+ 713us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:33.469075 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:33.469086 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:33.469116 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:23:33.469807 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:33.470449 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:33.471473 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:33.471779 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:33.471822 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:33.473577 (+ 1755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:23:33.473578 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:23:33.473579 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:23:33.473587 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:23:33.475378 (+ 1791us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:23:33.475808 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:23:33.475809 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:23:33.475810 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:23:33.475868 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:23:33.476095 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:23:33.476096 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:23:33.476096 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:23:33.476226 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:23:33.476296 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:33.476502 (+ 206us) server_negotiation.cc:299] Negotiation successful
0812 01:23:33.476593 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 01:23:33.478384 29225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:33.456736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54716 (local address 127.24.0.190:36549)
0812 01:23:33.456778 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:33.456780 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:33.468242 (+ 11462us) server_negotiation.cc:407] Connection header received
0812 01:23:33.468273 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:33.468274 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:33.468286 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:33.468337 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:23:33.469221 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:33.469920 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:33.473180 (+ 3260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:33.473491 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:33.473533 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:33.475223 (+ 1690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:23:33.475225 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:23:33.475225 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:23:33.475232 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:23:33.476954 (+ 1722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:23:33.477563 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:23:33.477564 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:23:33.477564 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:23:33.477617 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:23:33.477813 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:23:33.477814 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:23:33.477814 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:23:33.477932 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:23:33.477996 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:33.478217 (+ 221us) server_negotiation.cc:299] Negotiation successful
0812 01:23:33.478295 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
Aug 12 01:23:33 dist-test-slave-t3q3 krb5kdc[29087](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497813, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.189@KRBTEST.COM for kudu/127.24.0.188@KRBTEST.COM
I20260812 01:23:33.484211 29350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:33.473181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.188:33185 (local address 127.0.0.1:39188)
0812 01:23:33.473258 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0812 01:23:33.473267 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 01:23:33.473300 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:23:33.476932 (+ 3632us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:23:33.476934 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:23:33.477095 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:23:33.477139 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:23:33.477141 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:23:33.478027 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:23:33.478028 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:23:33.478862 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:23:33.478865 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:23:33.479351 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:23:33.479351 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:23:33.479409 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:33.479788 (+ 379us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:23:33.479793 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:23:33.481256 (+ 1463us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:23:33.483131 (+ 1875us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:23:33.483132 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:23:33.483132 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:23:33.483344 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:23:33.483544 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:23:33.483554 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:23:33.483554 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:23:33.483613 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:23:33.483883 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:23:33.483884 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:23:33.483928 (+ 44us) client_negotiation.cc:769] Sending connection context
0812 01:23:33.483995 (+ 67us) client_negotiation.cc:240] Negotiation successful
0812 01:23:33.484114 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260812 01:23:33.484561 29489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:33.473373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39188 (local address 127.24.0.188:33185)
0812 01:23:33.476763 (+ 3390us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:33.476764 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:33.476773 (+ 9us) server_negotiation.cc:407] Connection header received
0812 01:23:33.476790 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:33.476791 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:33.476799 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:33.476852 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:23:33.477222 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:33.477926 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:33.478960 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:33.479246 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:33.479303 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:33.481445 (+ 2142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:23:33.481446 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:23:33.481447 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:23:33.481454 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:23:33.483029 (+ 1575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:23:33.483422 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:23:33.483423 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:23:33.483423 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:23:33.483476 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:23:33.483679 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:23:33.483680 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:23:33.483680 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:23:33.483800 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:23:33.483857 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:33.484391 (+ 534us) server_negotiation.cc:299] Negotiation successful
0812 01:23:33.484461 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3375,"thread_start_us":90,"threads_started":1}
I20260812 01:23:33.486137 29345 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e451071ddd44da1a0e8a17e726f5df0: Bootstrap starting.
I20260812 01:23:33.487082 29345 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0e451071ddd44da1a0e8a17e726f5df0: Neither blocks nor log segments found. Creating new log.
I20260812 01:23:33.487414 29345 log.cc:824] T 00000000000000000000000000000000 P 0e451071ddd44da1a0e8a17e726f5df0: Log is configured to *not* fsync() on all Append() calls
I20260812 01:23:33.488358 29345 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e451071ddd44da1a0e8a17e726f5df0: No bootstrap required, opened a new log
I20260812 01:23:33.492587 29345 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0e451071ddd44da1a0e8a17e726f5df0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83c4efc95926420d8a7bf318f9dafd8f" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 36549 } } peers { permanent_uuid: "0e451071ddd44da1a0e8a17e726f5df0" member_type: VOTER last_known_addr { host: "127.24.0.189" port: 42977 } } peers { permanent_uuid: "e447e798d4d2499ebd1fe2c1ad7cbda4" member_type: VOTER last_known_addr { host: "127.24.0.188" port: 33185 } }
I20260812 01:23:33.492892 29345 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0e451071ddd44da1a0e8a17e726f5df0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e451071ddd44da1a0e8a17e726f5df0, State: Initialized, Role: FOLLOWER
I20260812 01:23:33.493108 29345 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0e451071ddd44da1a0e8a17e726f5df0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83c4efc95926420d8a7bf318f9dafd8f" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 36549 } } peers { permanent_uuid: "0e451071ddd44da1a0e8a17e726f5df0" member_type: VOTER last_known_addr { host: "127.24.0.189" port: 42977 } } peers { permanent_uuid: "e447e798d4d2499ebd1fe2c1ad7cbda4" member_type: VOTER last_known_addr { host: "127.24.0.188" port: 33185 } }
I20260812 01:23:33.493956 29302 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0e451071ddd44da1a0e8a17e726f5df0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:23:33.494848 29302 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0e451071ddd44da1a0e8a17e726f5df0 [term 1 FOLLOWER]: Refusing update from remote peer 83c4efc95926420d8a7bf318f9dafd8f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:23:33.493979 29303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "83c4efc95926420d8a7bf318f9dafd8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "0e451071ddd44da1a0e8a17e726f5df0"
I20260812 01:23:33.495148 29511 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 83c4efc95926420d8a7bf318f9dafd8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e451071ddd44da1a0e8a17e726f5df0" member_type: VOTER last_known_addr { host: "127.24.0.189" port: 42977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:23:33.495838 29303 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0e451071ddd44da1a0e8a17e726f5df0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83c4efc95926420d8a7bf318f9dafd8f in term 1.
I20260812 01:23:33.493885 29345 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0e451071ddd44da1a0e8a17e726f5df0 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:23:33.500869 29570 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e451071ddd44da1a0e8a17e726f5df0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "83c4efc95926420d8a7bf318f9dafd8f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83c4efc95926420d8a7bf318f9dafd8f" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 36549 } } peers { permanent_uuid: "0e451071ddd44da1a0e8a17e726f5df0" member_type: VOTER last_known_addr { host: "127.24.0.189" port: 42977 } } peers { permanent_uuid: "e447e798d4d2499ebd1fe2c1ad7cbda4" member_type: VOTER last_known_addr { host: "127.24.0.188" port: 33185 } } }
I20260812 01:23:33.500962 29570 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e451071ddd44da1a0e8a17e726f5df0 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 01:23:33.501016 29570 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e451071ddd44da1a0e8a17e726f5df0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 83c4efc95926420d8a7bf318f9dafd8f. Latest consensus state: current_term: 1 leader_uuid: "83c4efc95926420d8a7bf318f9dafd8f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83c4efc95926420d8a7bf318f9dafd8f" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 36549 } } peers { permanent_uuid: "0e451071ddd44da1a0e8a17e726f5df0" member_type: VOTER last_known_addr { host: "127.24.0.189" port: 42977 } } peers { permanent_uuid: "e447e798d4d2499ebd1fe2c1ad7cbda4" member_type: VOTER last_known_addr { host: "127.24.0.188" port: 33185 } } }
I20260812 01:23:33.501089 29570 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e451071ddd44da1a0e8a17e726f5df0 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 01:23:33.501129 29570 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e451071ddd44da1a0e8a17e726f5df0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "83c4efc95926420d8a7bf318f9dafd8f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83c4efc95926420d8a7bf318f9dafd8f" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 36549 } } peers { permanent_uuid: "0e451071ddd44da1a0e8a17e726f5df0" member_type: VOTER last_known_addr { host: "127.24.0.189" port: 42977 } } peers { permanent_uuid: "e447e798d4d2499ebd1fe2c1ad7cbda4" member_type: VOTER last_known_addr { host: "127.24.0.188" port: 33185 } } }
I20260812 01:23:33.501179 29570 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e451071ddd44da1a0e8a17e726f5df0 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 01:23:33.501626 29568 mvcc.cc:205] Tried to move back new op lower bound from 7317495043925270528 to 7317495043900182528. Current Snapshot: MvccSnapshot[applied={T|T < 7317495043925270528}]
I20260812 01:23:33.508711 29591 catalog_manager.cc:1259] Loaded cluster ID: f9a41c0981b74ed49ae759973845ff3b
I20260812 01:23:33.508809 29591 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0e451071ddd44da1a0e8a17e726f5df0: loading cluster ID for follower catalog manager: success
W20260812 01:23:33.508942 29591 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 0e451071ddd44da1a0e8a17e726f5df0: acquiring CA information for follower catalog manager: Not found: root CA entry not found
W20260812 01:23:33.509028 29591 catalog_manager.cc:873] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
I20260812 01:23:33.510039 29511 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83c4efc95926420d8a7bf318f9dafd8f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "83c4efc95926420d8a7bf318f9dafd8f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83c4efc95926420d8a7bf318f9dafd8f" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 36549 } } peers { permanent_uuid: "0e451071ddd44da1a0e8a17e726f5df0" member_type: VOTER last_known_addr { host: "127.24.0.189" port: 42977 } } peers { permanent_uuid: "e447e798d4d2499ebd1fe2c1ad7cbda4" member_type: VOTER last_known_addr { host: "127.24.0.188" port: 33185 } } }
I20260812 01:23:33.510123 29511 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83c4efc95926420d8a7bf318f9dafd8f [sys.catalog]: This master's current role is: LEADER
I20260812 01:23:33.512099 29548 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:23:33.512949 29548 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:23:33.520706 29548 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 83c4efc95926420d8a7bf318f9dafd8f: Generated new TSK 0
I20260812 01:23:33.520885 29548 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 01:23:33.553969 29489 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:23:33.552143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39204 (local address 127.24.0.188:33185)
0812 01:23:33.552203 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:33.552206 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:33.553234 (+ 1028us) server_negotiation.cc:407] Connection header received
0812 01:23:33.553368 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:33.553370 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:33.553386 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:33.553423 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:23:33.553899 (+ 476us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39204: BlockingRecv error: recv got EOF from 127.0.0.1:39204 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
W20260812 01:23:33.554750 29225 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:23:33.552536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54732 (local address 127.24.0.190:36549)
0812 01:23:33.552579 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:33.552581 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:33.554227 (+ 1646us) server_negotiation.cc:407] Connection header received
0812 01:23:33.554326 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:33.554327 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:33.554337 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:33.554367 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:23:33.554686 (+ 319us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54732: BlockingRecv error: recv got EOF from 127.0.0.1:54732 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260812 01:23:33.555497 29355 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:23:33.552884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58010 (local address 127.24.0.189:42977)
0812 01:23:33.552923 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:33.552926 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:33.554984 (+ 2058us) server_negotiation.cc:407] Connection header received
0812 01:23:33.555094 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:33.555095 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:33.555106 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:33.555135 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:23:33.555437 (+ 302us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58010: BlockingRecv error: recv got EOF from 127.0.0.1:58010 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260812 01:23:33.557562 29355 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:23:33.556851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58026 (local address 127.24.0.189:42977)
0812 01:23:33.556891 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:33.556893 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:33.557046 (+ 153us) server_negotiation.cc:407] Connection header received
0812 01:23:33.557153 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:33.557154 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:33.557171 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:33.557199 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:23:33.557506 (+ 307us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58026: BlockingRecv error: recv got EOF from 127.0.0.1:58026 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260812 01:23:33.560576 29355 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:23:33.559826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58034 (local address 127.24.0.189:42977)
0812 01:23:33.559864 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:33.559866 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:33.560052 (+ 186us) server_negotiation.cc:407] Connection header received
0812 01:23:33.560172 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:33.560173 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:33.560182 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:33.560210 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:23:33.560532 (+ 322us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58034: BlockingRecv error: recv got EOF from 127.0.0.1:58034 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260812 01:23:33.564451 29355 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:23:33.563694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58046 (local address 127.24.0.189:42977)
0812 01:23:33.563728 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:33.563730 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:33.563922 (+ 192us) server_negotiation.cc:407] Connection header received
0812 01:23:33.564014 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:33.564015 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:33.564024 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:33.564053 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:23:33.564412 (+ 359us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58046: BlockingRecv error: recv got EOF from 127.0.0.1:58046 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260812 01:23:33.570317 29355 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:23:33.569551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58058 (local address 127.24.0.189:42977)
0812 01:23:33.569586 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:33.569587 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:33.569837 (+ 250us) server_negotiation.cc:407] Connection header received
0812 01:23:33.569932 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:33.569933 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:33.569942 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:33.569971 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:23:33.570273 (+ 302us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58058: BlockingRecv error: recv got EOF from 127.0.0.1:58058 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260812 01:23:33.580727 29355 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:23:33.579799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58062 (local address 127.24.0.189:42977)
0812 01:23:33.579842 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:33.579844 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:33.580138 (+ 294us) server_negotiation.cc:407] Connection header received
0812 01:23:33.580250 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:33.580251 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:33.580262 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:33.580292 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:23:33.580673 (+ 381us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58062: BlockingRecv error: recv got EOF from 127.0.0.1:58062 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260812 01:23:33.599261 29355 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:23:33.598232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58078 (local address 127.24.0.189:42977)
0812 01:23:33.598280 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:33.598283 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:33.598613 (+ 330us) server_negotiation.cc:407] Connection header received
0812 01:23:33.598727 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:33.598728 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:33.598740 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:33.598771 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:23:33.599197 (+ 426us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58078: BlockingRecv error: recv got EOF from 127.0.0.1:58078 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260812 01:23:33.633913 29355 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:23:33.632886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58082 (local address 127.24.0.189:42977)
0812 01:23:33.632932 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:33.632935 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:33.633267 (+ 332us) server_negotiation.cc:407] Connection header received
0812 01:23:33.633376 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:33.633377 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:33.633390 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:33.633420 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:23:33.633850 (+ 430us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58082: BlockingRecv error: recv got EOF from 127.0.0.1:58082 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260812 01:23:33.700316 29355 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:23:33.699417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58098 (local address 127.24.0.189:42977)
0812 01:23:33.699526 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:33.699529 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:33.699640 (+ 111us) server_negotiation.cc:407] Connection header received
0812 01:23:33.699733 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:33.699734 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:33.699749 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:33.699822 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:23:33.700253 (+ 431us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58098: BlockingRecv error: recv got EOF from 127.0.0.1:58098 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
W20260812 01:23:33.830686 29355 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:23:33.829797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58110 (local address 127.24.0.189:42977)
0812 01:23:33.829914 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:33.829917 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:33.830025 (+ 108us) server_negotiation.cc:407] Connection header received
0812 01:23:33.830149 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:33.830151 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:33.830176 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:33.830248 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:23:33.830635 (+ 387us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58110: BlockingRecv error: recv got EOF from 127.0.0.1:58110 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
W20260812 01:23:34.089031 29355 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:23:34.088298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58116 (local address 127.24.0.189:42977)
0812 01:23:34.088360 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:34.088362 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:34.088390 (+ 28us) server_negotiation.cc:407] Connection header received
0812 01:23:34.088470 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:34.088471 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:34.088484 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:34.088562 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:23:34.088955 (+ 393us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58116: BlockingRecv error: recv got EOF from 127.0.0.1:58116 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
I20260812 01:23:34.445034 29507 catalog_manager.cc:1259] Loaded cluster ID: f9a41c0981b74ed49ae759973845ff3b
I20260812 01:23:34.445165 29507 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e447e798d4d2499ebd1fe2c1ad7cbda4: loading cluster ID for follower catalog manager: success
I20260812 01:23:34.445963 29507 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e447e798d4d2499ebd1fe2c1ad7cbda4: acquiring CA information for follower catalog manager: success
I20260812 01:23:34.446290 29507 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e447e798d4d2499ebd1fe2c1ad7cbda4: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 01:23:34.510200 29591 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 0e451071ddd44da1a0e8a17e726f5df0: acquiring CA information for follower catalog manager: success
I20260812 01:23:34.510535 29591 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 0e451071ddd44da1a0e8a17e726f5df0: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 01:23:34.605418 29708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:34.602374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58132 (local address 127.24.0.189:42977)
0812 01:23:34.602522 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:34.602525 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:34.602545 (+ 20us) server_negotiation.cc:407] Connection header received
0812 01:23:34.602601 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:34.602602 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:34.602670 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:34.602766 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:23:34.602929 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:34.603435 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:34.604270 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:34.604472 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:34.604558 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:34.604751 (+ 193us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:23:34.605090 (+ 339us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:23:34.605238 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:34.605285 (+ 47us) server_negotiation.cc:299] Negotiation successful
0812 01:23:34.605311 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":65,"threads_started":1}
I20260812 01:23:34.610143 29719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:34.607452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39212 (local address 127.24.0.188:33185)
0812 01:23:34.607564 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:34.607566 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:34.607575 (+ 9us) server_negotiation.cc:407] Connection header received
0812 01:23:34.607648 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:34.607649 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:34.607708 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:34.607806 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:23:34.607905 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:34.608476 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:34.609251 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:34.609486 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:34.609575 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:34.609764 (+ 189us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:23:34.609877 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:23:34.609977 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:34.610020 (+ 43us) server_negotiation.cc:299] Negotiation successful
0812 01:23:34.610043 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84,"thread_start_us":46,"threads_started":1}
I20260812 01:23:34.614686 29708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:34.611789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58144 (local address 127.24.0.189:42977)
0812 01:23:34.611829 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:34.611831 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:34.612086 (+ 255us) server_negotiation.cc:407] Connection header received
0812 01:23:34.612183 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:34.612184 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:34.612196 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:34.612257 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:23:34.612355 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:34.612971 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:34.613876 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:34.614182 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:34.614221 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:34.614416 (+ 195us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:23:34.614452 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:23:34.614507 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:34.614587 (+ 80us) server_negotiation.cc:299] Negotiation successful
0812 01:23:34.614615 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 01:23:34.615125 29719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:34.611852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39218 (local address 127.24.0.188:33185)
0812 01:23:34.611997 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:34.611999 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:34.612479 (+ 480us) server_negotiation.cc:407] Connection header received
0812 01:23:34.612505 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:34.612505 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:34.612515 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:34.612561 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:23:34.612739 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:34.613466 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:34.614260 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:34.614560 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:34.614620 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:34.614821 (+ 201us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:23:34.614852 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:23:34.614924 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:34.615019 (+ 95us) server_negotiation.cc:299] Negotiation successful
0812 01:23:34.615039 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125}
I20260812 01:23:34.615581 29725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:34.611801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54748 (local address 127.24.0.190:36549)
0812 01:23:34.611893 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:34.611895 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:34.612020 (+ 125us) server_negotiation.cc:407] Connection header received
0812 01:23:34.612148 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:34.612149 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:34.612231 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:34.612317 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:23:34.612596 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:34.613429 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:34.614584 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:34.614916 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:34.614958 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:34.615209 (+ 251us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:23:34.615321 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:23:34.615363 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:34.615467 (+ 104us) server_negotiation.cc:299] Negotiation successful
0812 01:23:34.615492 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72,"thread_start_us":46,"threads_started":1}
I20260812 01:23:34.617390 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 29109
I20260812 01:23:34.633745 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 29231
I20260812 01:23:34.646153 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 29363
2026-08-12T01:23:34Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3100 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786497534195564-24578-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 01:23:34 dist-test-slave-t3q3 krb5kdc[29733](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 01:23:34 dist-test-slave-t3q3 krb5kdc[29733](info): set up 2 sockets
Aug 12 01:23:34 dist-test-slave-t3q3 krb5kdc[29733](info): commencing operation
krb5kdc: starting...
W20260812 01:23:35.940841 24578 mini_kdc.cc:121] Time spent starting KDC: real 1.269s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786497534195564-24578-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786497534195564-24578-0/krb5kdc/test-user.keytab.
Aug 12 01:23:36 dist-test-slave-t3q3 krb5kdc[29733](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497816, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T01:23:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:23:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.0.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.0.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.0.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab.
Entry for principal kudu/127.24.0.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab.
I20260812 01:23:36.070749 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786497534195564-24578-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786497534195564-24578-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786497534195564-24578-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.190:37259
--webserver_interface=127.24.0.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab
--principal=kudu/127.24.0.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:45675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.0.190:37259,127.24.0.189:38941,127.24.0.188:32789
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:23:36.153374 29756 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 01:23:36.153649 29756 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:23:36.153681 29756 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:23:36.153714 29756 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:23:36.155073 29756 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:23:36.155171 29756 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:23:36.155200 29756 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:23:36.155232 29756 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:23:36.155251 29756 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:23:36.156790 29756 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:45675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786497534195564-24578-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786497534195564-24578-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.0.190:37259,127.24.0.189:38941,127.24.0.188:32789
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.190.keytab
--principal=kudu/127.24.0.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.190:37259
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.0.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29756
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786497534195564-24578-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:23:36.157119 29756 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:23:36.157364 29756 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:23:36.162499 29762 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:23:36.162563 29763 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:23:36.162545 29767 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:23:36.162986 29756 server_base.cc:1034] running on GCE node
I20260812 01:23:36.163264 29756 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:23:36.163535 29756 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:23:36.164707 29756 hybrid_clock.cc:648] HybridClock initialized: now 1786497816164696 us; error 33 us; skew 500 ppm
Aug 12 01:23:36 dist-test-slave-t3q3 krb5kdc[29733](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497816, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:23:36.167412 29756 init.cc:377] Logged in from keytab as kudu/127.24.0.190@KRBTEST.COM (short username kudu)
I20260812 01:23:36.167888 29756 webserver.cc:460] Webserver started at http://127.24.0.190:37329/ using document root <none> and password file <none>
I20260812 01:23:36.168170 29756 fs_manager.cc:359] Metadata directory not provided
I20260812 01:23:36.168220 29756 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:23:36.168318 29756 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:23:36.169364 29756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786497534195564-24578-0/minicluster-data/master-0/data/instance:
uuid: "84f90d3c789a459a9b28b36b6ddf155b"
format_stamp: "Formatted at 2026-08-12 01:23:36 on dist-test-slave-t3q3"
server_key: "1d445b697976d30f94b8f2c771f43308"
server_key_iv: "655325b6b114a8e91506817eca0a0a76"
server_key_version: "encryptionkey@0"
I20260812 01:23:36.169680 29756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786497534195564-24578-0/minicluster-data/master-0/wal/instance:
uuid: "84f90d3c789a459a9b28b36b6ddf155b"
format_stamp: "Formatted at 2026-08-12 01:23:36 on dist-test-slave-t3q3"
server_key: "1d445b697976d30f94b8f2c771f43308"
server_key_iv: "655325b6b114a8e91506817eca0a0a76"
server_key_version: "encryptionkey@0"
I20260812 01:23:36.171006 29756 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 01:23:36.171810 29778 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:23:36.172163 29756 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:23:36.172305 29756 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786497534195564-24578-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786497534195564-24578-0/minicluster-data/master-0/wal
uuid: "84f90d3c789a459a9b28b36b6ddf155b"
format_stamp: "Formatted at 2026-08-12 01:23:36 on dist-test-slave-t3q3"
server_key: "1d445b697976d30f94b8f2c771f43308"
server_key_iv: "655325b6b114a8e91506817eca0a0a76"
server_key_version: "encryptionkey@0"
I20260812 01:23:36.172395 29756 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786497534195564-24578-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786497534195564-24578-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786497534195564-24578-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:23:36.194809 29756 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:23:36.197239 29756 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:23:36.197415 29756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:23:36.201931 29756 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.190:37259
I20260812 01:23:36.201982 29851 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.190:37259 every 8 connection(s)
I20260812 01:23:36.202373 29756 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786497534195564-24578-0/minicluster-data/master-0/data/info.pb
I20260812 01:23:36.203382 29854 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:23:36.205307 29854 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.0.190" port: 37259 } has no permanent_uuid. Determining permanent_uuid...
I20260812 01:23:36.206358 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 29756
I20260812 01:23:36.206491 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786497534195564-24578-0/minicluster-data/master-0/wal/instance
I20260812 01:23:36.206727 24578 external_mini_cluster.cc:1428] Setting key 376e7143535cf925be92d8ed5bde1922
Aug 12 01:23:36 dist-test-slave-t3q3 krb5kdc[29733](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497816, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.190@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.24.0.189@KRBTEST.COM; defaulting to no policy
I20260812 01:23:36.217123 29857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:36.206065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:37259 (local address 127.0.0.1:46036)
0812 01:23:36.206319 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0812 01:23:36.206358 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 01:23:36.206990 (+ 632us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:23:36.207373 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:23:36.207376 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:23:36.207652 (+ 276us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:23:36.207869 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:23:36.207874 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:23:36.209271 (+ 1397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:23:36.209273 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:23:36.210381 (+ 1108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:23:36.210384 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:23:36.210919 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:23:36.210921 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:23:36.211011 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:36.211649 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:23:36.211660 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:23:36.213392 (+ 1732us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:23:36.215356 (+ 1964us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:23:36.215363 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:23:36.215366 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:23:36.215579 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:23:36.215895 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:23:36.215895 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:23:36.215896 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:23:36.215992 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:23:36.216445 (+ 453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:23:36.216451 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:23:36.216520 (+ 69us) client_negotiation.cc:769] Sending connection context
0812 01:23:36.216646 (+ 126us) client_negotiation.cc:240] Negotiation successful
0812 01:23:36.216923 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"mutex_wait_us":79,"thread_start_us":149,"threads_started":1}
I20260812 01:23:36.217123 29858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:36.206346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46036 (local address 127.24.0.190:37259)
0812 01:23:36.206535 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:36.206540 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:36.206578 (+ 38us) server_negotiation.cc:407] Connection header received
0812 01:23:36.207175 (+ 597us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:36.207178 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:36.207212 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:36.207314 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:23:36.207993 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:36.209159 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:36.210481 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:36.210797 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:36.210880 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:36.213498 (+ 2618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:23:36.213506 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:23:36.213507 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:23:36.213514 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:23:36.215233 (+ 1719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:23:36.215698 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:23:36.215701 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:23:36.215701 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:23:36.215827 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:23:36.216123 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:23:36.216123 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:23:36.216124 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:23:36.216360 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:23:36.216466 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:36.216692 (+ 226us) server_negotiation.cc:299] Negotiation successful
0812 01:23:36.216854 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":86,"threads_started":1}
I20260812 01:23:36.218037 29854 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.0.189" port: 38941 } has no permanent_uuid. Determining permanent_uuid...
I20260812 01:23:36.218479 29857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:36.218322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.189:38941 (local address 127.0.0.1:34474)
0812 01:23:36.218373 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0812 01:23:36.218443 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.189:38941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.0.189@KRBTEST.COM" created.
W20260812 01:23:36.218642 29782 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.0.189:38941: connect: Connection refused (error 111)
I20260812 01:23:36.219129 29857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:36.218990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.189:38941 (local address 127.0.0.1:34480)
0812 01:23:36.219048 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0812 01:23:36.219075 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.189:38941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260812 01:23:36.219291 29854 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.0.189:38941: Network error: Client connection negotiation failed: client connection to 127.24.0.189:38941: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.0.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.189.keytab.
Entry for principal kudu/127.24.0.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.189.keytab.
I20260812 01:23:36.235678 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786497534195564-24578-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786497534195564-24578-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786497534195564-24578-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786497534195564-24578-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.189:38941
--webserver_interface=127.24.0.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.189.keytab
--principal=kudu/127.24.0.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:45675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.0.190:37259,127.24.0.189:38941,127.24.0.188:32789
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 01:23:36.268554 29854 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.0.189" port: 38941 } attempt: 1
I20260812 01:23:36.269207 29857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:36.269033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.189:38941 (local address 127.0.0.1:34482)
0812 01:23:36.269114 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0812 01:23:36.269156 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.189:38941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260812 01:23:36.269605 29857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:36.269497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.189:38941 (local address 127.0.0.1:34488)
0812 01:23:36.269553 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0812 01:23:36.269579 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.189:38941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260812 01:23:36.269726 29854 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.0.189:38941: Network error: Client connection negotiation failed: client connection to 127.24.0.189:38941: connect: Connection refused (error 111)
W20260812 01:23:36.321965 29866 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 01:23:36.322237 29866 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:23:36.322270 29866 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:23:36.322311 29866 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:23:36.323460 29866 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:23:36.323545 29866 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:23:36.323571 29866 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:23:36.323601 29866 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:23:36.323632 29866 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:23:36.325242 29866 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:45675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786497534195564-24578-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786497534195564-24578-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.24.0.190:37259,127.24.0.189:38941,127.24.0.188:32789
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.189.keytab
--principal=kudu/127.24.0.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.189:38941
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786497534195564-24578-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.24.0.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29866
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786497534195564-24578-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:23:36.325548 29866 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:23:36.325809 29866 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:23:36.330917 29877 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:23:36.330914 29880 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:23:36.331173 29866 server_base.cc:1034] running on GCE node
W20260812 01:23:36.331284 29876 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:23:36.331571 29866 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:23:36.331893 29866 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:23:36.333084 29866 hybrid_clock.cc:648] HybridClock initialized: now 1786497816333066 us; error 54 us; skew 500 ppm
Aug 12 01:23:36 dist-test-slave-t3q3 krb5kdc[29733](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497816, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:23:36.335812 29866 init.cc:377] Logged in from keytab as kudu/127.24.0.189@KRBTEST.COM (short username kudu)
I20260812 01:23:36.336374 29866 webserver.cc:460] Webserver started at http://127.24.0.189:37589/ using document root <none> and password file <none>
I20260812 01:23:36.336670 29866 fs_manager.cc:359] Metadata directory not provided
I20260812 01:23:36.336751 29866 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:23:36.336915 29866 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:23:36.337891 29854 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.0.189" port: 38941 } attempt: 2
I20260812 01:23:36.338131 29866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786497534195564-24578-0/minicluster-data/master-1/data/instance:
uuid: "b35fca45a01942b386f5656876b7d5ce"
format_stamp: "Formatted at 2026-08-12 01:23:36 on dist-test-slave-t3q3"
server_key: "618fe168d4ee70ad9b83005c32fe04b7"
server_key_iv: "c2ca445f37b3bfbb4e582db06c3ad09d"
server_key_version: "encryptionkey@0"
I20260812 01:23:36.338312 29857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:36.338174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.189:38941 (local address 127.0.0.1:34500)
0812 01:23:36.338244 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0812 01:23:36.338281 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.189:38941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260812 01:23:36.338614 29866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786497534195564-24578-0/minicluster-data/master-1/wal/instance:
uuid: "b35fca45a01942b386f5656876b7d5ce"
format_stamp: "Formatted at 2026-08-12 01:23:36 on dist-test-slave-t3q3"
server_key: "618fe168d4ee70ad9b83005c32fe04b7"
server_key_iv: "c2ca445f37b3bfbb4e582db06c3ad09d"
server_key_version: "encryptionkey@0"
I20260812 01:23:36.338780 29857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:36.338646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.189:38941 (local address 127.0.0.1:34506)
0812 01:23:36.338720 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0812 01:23:36.338752 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.189:38941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260812 01:23:36.338907 29854 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.0.189:38941: Network error: Client connection negotiation failed: client connection to 127.24.0.189:38941: connect: Connection refused (error 111)
I20260812 01:23:36.340147 29866 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260812 01:23:36.340886 29895 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:23:36.341090 29866 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:23:36.341198 29866 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786497534195564-24578-0/minicluster-data/master-1/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786497534195564-24578-0/minicluster-data/master-1/wal
uuid: "b35fca45a01942b386f5656876b7d5ce"
format_stamp: "Formatted at 2026-08-12 01:23:36 on dist-test-slave-t3q3"
server_key: "618fe168d4ee70ad9b83005c32fe04b7"
server_key_iv: "c2ca445f37b3bfbb4e582db06c3ad09d"
server_key_version: "encryptionkey@0"
I20260812 01:23:36.341292 29866 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786497534195564-24578-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786497534195564-24578-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786497534195564-24578-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:23:36.357138 29866 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:23:36.359754 29866 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:23:36.359941 29866 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:23:36.364014 29866 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.189:38941
I20260812 01:23:36.364120 29979 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.189:38941 every 8 connection(s)
I20260812 01:23:36.364497 29866 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786497534195564-24578-0/minicluster-data/master-1/data/info.pb
I20260812 01:23:36.365536 29981 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:23:36.367192 29981 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.0.190" port: 37259 } has no permanent_uuid. Determining permanent_uuid...
I20260812 01:23:36.372252 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 29866
I20260812 01:23:36.372357 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786497534195564-24578-0/minicluster-data/master-1/wal/instance
I20260812 01:23:36.372583 24578 external_mini_cluster.cc:1428] Setting key 4ba5cb42fec45a87b1a92a7618d42e9d
Aug 12 01:23:36 dist-test-slave-t3q3 krb5kdc[29733](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497816, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.189@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
I20260812 01:23:36.377185 29983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:36.367905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:37259 (local address 127.0.0.1:46048)
0812 01:23:36.368128 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0812 01:23:36.368179 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 01:23:36.368632 (+ 453us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:23:36.368891 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:23:36.368894 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:23:36.369169 (+ 275us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:23:36.369368 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:23:36.369372 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:23:36.370069 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:23:36.370070 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:23:36.370884 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:23:36.370886 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:23:36.371261 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:23:36.371262 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:23:36.371377 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:36.371891 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:23:36.371900 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:23:36.373887 (+ 1987us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:23:36.375706 (+ 1819us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:23:36.375713 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:23:36.375715 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:23:36.375994 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:23:36.376275 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:23:36.376276 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:23:36.376277 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:23:36.376394 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:23:36.376701 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:23:36.376706 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:23:36.376820 (+ 114us) client_negotiation.cc:769] Sending connection context
0812 01:23:36.376903 (+ 83us) client_negotiation.cc:240] Negotiation successful
0812 01:23:36.377020 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":109,"threads_started":1}
I20260812 01:23:36.377200 29858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:36.367984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46048 (local address 127.24.0.190:37259)
0812 01:23:36.368129 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:36.368132 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:36.368253 (+ 121us) server_negotiation.cc:407] Connection header received
0812 01:23:36.368727 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:36.368728 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:36.368740 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:36.368805 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:23:36.369467 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:36.369961 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:36.370955 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:36.371166 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:36.371224 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:36.373988 (+ 2764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:23:36.373989 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:23:36.373990 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:23:36.373996 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:23:36.375619 (+ 1623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:23:36.376111 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:23:36.376112 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:23:36.376113 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:23:36.376181 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:23:36.376473 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:23:36.376474 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:23:36.376474 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:23:36.376628 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:23:36.376700 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:36.376958 (+ 258us) server_negotiation.cc:299] Negotiation successful
0812 01:23:36.377059 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 01:23:36.377970 29981 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.0.189" port: 38941 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.24.0.188@KRBTEST.COM; defaulting to no policy
Aug 12 01:23:36 dist-test-slave-t3q3 krb5kdc[29733](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497816, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.189@KRBTEST.COM for kudu/127.24.0.189@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.0.188@KRBTEST.COM" created.
I20260812 01:23:36.386170 29983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:36.378274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.189:38941 (local address 127.0.0.1:34516)
0812 01:23:36.378340 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0812 01:23:36.378352 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 01:23:36.378397 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:23:36.378639 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:23:36.378640 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:23:36.378812 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:23:36.378879 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:23:36.378880 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:23:36.379728 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:23:36.379729 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:23:36.380589 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:23:36.380592 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:23:36.381066 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:23:36.381067 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:23:36.381151 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:36.381520 (+ 369us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:23:36.381525 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:23:36.382925 (+ 1400us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:23:36.384833 (+ 1908us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:23:36.384834 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:23:36.384834 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:23:36.385111 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:23:36.385407 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:23:36.385408 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:23:36.385408 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:23:36.385487 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:23:36.385796 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:23:36.385797 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:23:36.385837 (+ 40us) client_negotiation.cc:769] Sending connection context
0812 01:23:36.385935 (+ 98us) client_negotiation.cc:240] Negotiation successful
0812 01:23:36.386050 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260812 01:23:36.386235 29985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:36.378313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34516 (local address 127.24.0.189:38941)
0812 01:23:36.378480 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:36.378484 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:36.378502 (+ 18us) server_negotiation.cc:407] Connection header received
0812 01:23:36.378531 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:36.378533 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:36.378543 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:36.378605 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:23:36.378959 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:36.379626 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:36.380676 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:36.380969 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:36.381026 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:36.383010 (+ 1984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:23:36.383018 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:23:36.383018 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:23:36.383024 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:23:36.384728 (+ 1704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:23:36.385235 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:23:36.385239 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:23:36.385239 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:23:36.385304 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:23:36.385563 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:23:36.385564 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:23:36.385564 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:23:36.385707 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:23:36.385817 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:36.385972 (+ 155us) server_negotiation.cc:299] Negotiation successful
0812 01:23:36.386106 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":66,"server-negotiator.queue_time_us":138,"thread_start_us":111,"threads_started":1}
I20260812 01:23:36.386828 29981 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.0.188" port: 32789 } has no permanent_uuid. Determining permanent_uuid...
I20260812 01:23:36.387207 29983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:36.387066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.188:32789 (local address 127.0.0.1:35738)
0812 01:23:36.387124 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0812 01:23:36.387175 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.188:32789: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260812 01:23:36.387322 29897 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.0.188:32789: connect: Connection refused (error 111)
I20260812 01:23:36.387799 29983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:36.387686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.188:32789 (local address 127.0.0.1:35746)
0812 01:23:36.387743 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0812 01:23:36.387774 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.188:32789: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260812 01:23:36.387892 29981 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.0.188:32789: Network error: Client connection negotiation failed: client connection to 127.24.0.188:32789: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.0.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.188.keytab.
Entry for principal kudu/127.24.0.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.188.keytab.
I20260812 01:23:36.404402 24578 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoD5sF/build/release/bin/kudu
/tmp/dist-test-taskvoD5sF/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786497534195564-24578-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786497534195564-24578-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786497534195564-24578-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786497534195564-24578-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.188:32789
--webserver_interface=127.24.0.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.188.keytab
--principal=kudu/127.24.0.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.0.148:45675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.0.190:37259,127.24.0.189:38941,127.24.0.188:32789
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786497534195564-24578-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786497534195564-24578-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786497534195564-24578-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 01:23:36.430053 29854 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.0.189" port: 38941 } attempt: 3
Aug 12 01:23:36 dist-test-slave-t3q3 krb5kdc[29733](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497816, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.190@KRBTEST.COM for kudu/127.24.0.189@KRBTEST.COM
I20260812 01:23:36.437098 29981 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.0.188" port: 32789 } attempt: 1
I20260812 01:23:36.437491 29983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:36.437356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.188:32789 (local address 127.0.0.1:35760)
0812 01:23:36.437421 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0812 01:23:36.437451 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.188:32789: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260812 01:23:36.437836 29983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:36.437719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.188:32789 (local address 127.0.0.1:35776)
0812 01:23:36.437779 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0812 01:23:36.437806 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.188:32789: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260812 01:23:36.437913 29981 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.0.188:32789: Network error: Client connection negotiation failed: client connection to 127.24.0.188:32789: connect: Connection refused (error 111)
I20260812 01:23:36.438109 29857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:36.430486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.189:38941 (local address 127.0.0.1:34524)
0812 01:23:36.430536 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0812 01:23:36.430552 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 01:23:36.430602 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:23:36.430817 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:23:36.430818 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:23:36.430989 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:23:36.431065 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:23:36.431067 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:23:36.431768 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:23:36.431769 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:23:36.432365 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:23:36.432371 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:23:36.432793 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:23:36.432794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:23:36.432883 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:36.433404 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:23:36.433410 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:23:36.434864 (+ 1454us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:23:36.436630 (+ 1766us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:23:36.436631 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:23:36.436632 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:23:36.436812 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:23:36.437044 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:23:36.437045 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:23:36.437045 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:23:36.437121 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:23:36.437486 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:23:36.437487 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:23:36.437546 (+ 59us) client_negotiation.cc:769] Sending connection context
0812 01:23:36.437828 (+ 282us) client_negotiation.cc:240] Negotiation successful
0812 01:23:36.438014 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23}
I20260812 01:23:36.438136 29985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:36.430586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34524 (local address 127.24.0.189:38941)
0812 01:23:36.430672 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:36.430675 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:36.430684 (+ 9us) server_negotiation.cc:407] Connection header received
0812 01:23:36.430719 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:36.430721 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:36.430731 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:36.430796 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:23:36.431157 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:36.431662 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:36.432463 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:36.432705 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:36.432795 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:36.434960 (+ 2165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:23:36.434961 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:23:36.434962 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:23:36.434970 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:23:36.436518 (+ 1548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:23:36.436894 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:23:36.436895 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:23:36.436895 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:23:36.436958 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:23:36.437245 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:23:36.437246 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:23:36.437247 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:23:36.437406 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:23:36.437493 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:36.437949 (+ 456us) server_negotiation.cc:299] Negotiation successful
0812 01:23:36.438045 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 01:23:36.438585 29854 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.0.188" port: 32789 } has no permanent_uuid. Determining permanent_uuid...
I20260812 01:23:36.438946 29857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:36.438823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.188:32789 (local address 127.0.0.1:35792)
0812 01:23:36.438886 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0812 01:23:36.438920 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.188:32789: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260812 01:23:36.439287 29857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:36.439186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.188:32789 (local address 127.0.0.1:35808)
0812 01:23:36.439241 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0812 01:23:36.439269 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.188:32789: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260812 01:23:36.439392 29854 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.0.188:32789: Network error: Client connection negotiation failed: client connection to 127.24.0.188:32789: connect: Connection refused (error 111)
I20260812 01:23:36.470589 29854 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.0.188" port: 32789 } attempt: 1
I20260812 01:23:36.471249 29857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:36.471068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.188:32789 (local address 127.0.0.1:35820)
0812 01:23:36.471154 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0812 01:23:36.471209 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.188:32789: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260812 01:23:36.471671 29857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:36.471558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.188:32789 (local address 127.0.0.1:35822)
0812 01:23:36.471612 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0812 01:23:36.471645 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.188:32789: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260812 01:23:36.471789 29854 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.0.188:32789: Network error: Client connection negotiation failed: client connection to 127.24.0.188:32789: connect: Connection refused (error 111)
W20260812 01:23:36.491276 29991 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 01:23:36.491492 29991 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:23:36.491510 29991 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:23:36.491570 29991 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:23:36.493014 29991 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:23:36.493077 29991 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:23:36.493098 29991 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:23:36.493122 29991 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:23:36.493144 29991 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:23:36.495702 29991 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.0.148:45675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786497534195564-24578-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786497534195564-24578-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.24.0.190:37259,127.24.0.189:38941,127.24.0.188:32789
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786497534195564-24578-0/krb5kdc/kudu_127.24.0.188.keytab
--principal=kudu/127.24.0.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.0.188:32789
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786497534195564-24578-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.24.0.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29991
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786497534195564-24578-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:23:36.495998 29991 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:23:36.496342 29991 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:23:36.502640 29997 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:23:36.504004 30002 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:23:36.504282 29998 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:23:36.504657 29991 server_base.cc:1034] running on GCE node
I20260812 01:23:36.504966 29991 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:23:36.505343 29991 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:23:36.506048 29981 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.0.188" port: 32789 } attempt: 2
I20260812 01:23:36.506508 29983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:36.506351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.188:32789 (local address 127.0.0.1:35832)
0812 01:23:36.506406 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0812 01:23:36.506449 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.188:32789: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260812 01:23:36.506536 29991 hybrid_clock.cc:648] HybridClock initialized: now 1786497816506500 us; error 46 us; skew 500 ppm
I20260812 01:23:36.506851 29983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:36.506778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.188:32789 (local address 127.0.0.1:35844)
0812 01:23:36.506812 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0812 01:23:36.506836 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.0.188:32789: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
W20260812 01:23:36.506981 29981 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.0.188:32789: Network error: Client connection negotiation failed: client connection to 127.24.0.188:32789: connect: Connection refused (error 111)
Aug 12 01:23:36 dist-test-slave-t3q3 krb5kdc[29733](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497816, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:23:36.509384 29991 init.cc:377] Logged in from keytab as kudu/127.24.0.188@KRBTEST.COM (short username kudu)
I20260812 01:23:36.509865 29991 webserver.cc:460] Webserver started at http://127.24.0.188:43129/ using document root <none> and password file <none>
I20260812 01:23:36.510056 29991 fs_manager.cc:359] Metadata directory not provided
I20260812 01:23:36.510097 29991 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:23:36.510207 29991 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:23:36.511257 29991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786497534195564-24578-0/minicluster-data/master-2/data/instance:
uuid: "4bcab980295242c9962ef9088dd6c5f9"
format_stamp: "Formatted at 2026-08-12 01:23:36 on dist-test-slave-t3q3"
server_key: "3bab86015c895b9e26f0e8c452a26c65"
server_key_iv: "e84f104337353d9f9b7cd494cf8e96ae"
server_key_version: "encryptionkey@0"
I20260812 01:23:36.511567 29991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786497534195564-24578-0/minicluster-data/master-2/wal/instance:
uuid: "4bcab980295242c9962ef9088dd6c5f9"
format_stamp: "Formatted at 2026-08-12 01:23:36 on dist-test-slave-t3q3"
server_key: "3bab86015c895b9e26f0e8c452a26c65"
server_key_iv: "e84f104337353d9f9b7cd494cf8e96ae"
server_key_version: "encryptionkey@0"
I20260812 01:23:36.513084 29991 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 01:23:36.513813 30017 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:23:36.514026 29991 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 01:23:36.514102 29991 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786497534195564-24578-0/minicluster-data/master-2/data,/tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786497534195564-24578-0/minicluster-data/master-2/wal
uuid: "4bcab980295242c9962ef9088dd6c5f9"
format_stamp: "Formatted at 2026-08-12 01:23:36 on dist-test-slave-t3q3"
server_key: "3bab86015c895b9e26f0e8c452a26c65"
server_key_iv: "e84f104337353d9f9b7cd494cf8e96ae"
server_key_version: "encryptionkey@0"
I20260812 01:23:36.514225 29991 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786497534195564-24578-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786497534195564-24578-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786497534195564-24578-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:23:36.525223 29991 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:23:36.527062 29991 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:23:36.527222 29991 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:23:36.531477 29991 rpc_server.cc:307] RPC server started. Bound to: 127.24.0.188:32789
I20260812 01:23:36.531538 30097 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.0.188:32789 every 8 connection(s)
I20260812 01:23:36.531924 29991 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786497534195564-24578-0/minicluster-data/master-2/data/info.pb
I20260812 01:23:36.532302 24578 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoD5sF/build/release/bin/kudu as pid 29991
I20260812 01:23:36.532431 24578 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoD5sF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786497534195564-24578-0/minicluster-data/master-2/wal/instance
I20260812 01:23:36.532644 24578 external_mini_cluster.cc:1428] Setting key 1181ac2b76a371b40cdac2ee7888464f
I20260812 01:23:36.533375 30099 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:23:36.535483 30099 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.0.190" port: 37259 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 01:23:36 dist-test-slave-t3q3 krb5kdc[29733](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497816, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.0.188@KRBTEST.COM
Aug 12 01:23:36 dist-test-slave-t3q3 krb5kdc[29733](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497816, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.188@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
I20260812 01:23:36.543617 30104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:36.533224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35858 (local address 127.24.0.188:32789)
0812 01:23:36.533441 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:36.533451 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:36.533483 (+ 32us) server_negotiation.cc:407] Connection header received
0812 01:23:36.534064 (+ 581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:36.534073 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:36.534137 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:36.534298 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:23:36.534899 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:36.536301 (+ 1402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:36.537167 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:36.537571 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:36.537721 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:36.540467 (+ 2746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:23:36.540473 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:23:36.540474 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:23:36.540480 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:23:36.542061 (+ 1581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:23:36.542457 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:23:36.542460 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:23:36.542461 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:23:36.542551 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:23:36.542795 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:23:36.542796 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:23:36.542796 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:23:36.543024 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:23:36.543204 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:36.543314 (+ 110us) server_negotiation.cc:299] Negotiation successful
0812 01:23:36.543444 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":100,"threads_started":1}
I20260812 01:23:36.544026 30106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:36.536227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.190:37259 (local address 127.0.0.1:46052)
0812 01:23:36.536377 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0812 01:23:36.536415 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 01:23:36.536484 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:23:36.536637 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:23:36.536639 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:23:36.536898 (+ 259us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:23:36.536965 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:23:36.536967 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:23:36.537621 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:23:36.537623 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:23:36.538210 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:23:36.538213 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:23:36.538652 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:23:36.538653 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:23:36.538728 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:36.539250 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:23:36.539262 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:23:36.541008 (+ 1746us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:23:36.542805 (+ 1797us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:23:36.542812 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:23:36.542814 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:23:36.543022 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:23:36.543263 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:23:36.543264 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:23:36.543264 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:23:36.543328 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:23:36.543613 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:23:36.543620 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:23:36.543670 (+ 50us) client_negotiation.cc:769] Sending connection context
0812 01:23:36.543795 (+ 125us) client_negotiation.cc:240] Negotiation successful
0812 01:23:36.543906 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":96,"threads_started":1}
I20260812 01:23:36.544046 29858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:36.536252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46052 (local address 127.24.0.190:37259)
0812 01:23:36.536333 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:36.536335 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:36.536470 (+ 135us) server_negotiation.cc:407] Connection header received
0812 01:23:36.536549 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:36.536550 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:36.536561 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:36.536619 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:23:36.537067 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:36.537523 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:36.538324 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:36.538526 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:36.538617 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:36.541111 (+ 2494us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:23:36.541113 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:23:36.541113 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:23:36.541121 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:23:36.542692 (+ 1571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:23:36.543118 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:23:36.543119 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:23:36.543120 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:23:36.543180 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:23:36.543414 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:23:36.543415 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:23:36.543415 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:23:36.543560 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:23:36.543621 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:36.543839 (+ 218us) server_negotiation.cc:299] Negotiation successful
0812 01:23:36.543931 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 01:23:36.544627 30099 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.0.189" port: 38941 } has no permanent_uuid. Determining permanent_uuid...
I20260812 01:23:36.546938 29854 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.0.188" port: 32789 } attempt: 2
Aug 12 01:23:36 dist-test-slave-t3q3 krb5kdc[29733](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497816, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.188@KRBTEST.COM for kudu/127.24.0.189@KRBTEST.COM
Aug 12 01:23:36 dist-test-slave-t3q3 krb5kdc[29733](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497816, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.0.190@KRBTEST.COM
Aug 12 01:23:36 dist-test-slave-t3q3 krb5kdc[29733](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497816, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.190@KRBTEST.COM for kudu/127.24.0.188@KRBTEST.COM
I20260812 01:23:36.552636 30106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:36.544895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.189:38941 (local address 127.0.0.1:34538)
0812 01:23:36.545065 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0812 01:23:36.545076 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 01:23:36.545125 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:23:36.545278 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:23:36.545279 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:23:36.545392 (+ 113us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:23:36.545436 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:23:36.545437 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:23:36.546109 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:23:36.546110 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:23:36.546591 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:23:36.546593 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:23:36.547007 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:23:36.547008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:23:36.547084 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:36.547532 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:23:36.547537 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:23:36.548789 (+ 1252us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:23:36.551344 (+ 2555us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:23:36.551345 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:23:36.551346 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:23:36.551572 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:23:36.551835 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:23:36.551836 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:23:36.551836 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:23:36.551906 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:23:36.552311 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:23:36.552312 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:23:36.552358 (+ 46us) client_negotiation.cc:769] Sending connection context
0812 01:23:36.552440 (+ 82us) client_negotiation.cc:240] Negotiation successful
0812 01:23:36.552529 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151}
I20260812 01:23:36.552709 29985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:36.545002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34538 (local address 127.24.0.189:38941)
0812 01:23:36.545085 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:36.545087 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:36.545139 (+ 52us) server_negotiation.cc:407] Connection header received
0812 01:23:36.545198 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:36.545199 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:36.545212 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:36.545276 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:23:36.545514 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:36.546015 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:36.546681 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:36.546909 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:36.546996 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:36.548888 (+ 1892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:23:36.548889 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:23:36.548890 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:23:36.548898 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:23:36.551239 (+ 2341us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:23:36.551678 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:23:36.551680 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:23:36.551680 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:23:36.551747 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:23:36.552033 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:23:36.552035 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:23:36.552035 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:23:36.552203 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:23:36.552278 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:36.552500 (+ 222us) server_negotiation.cc:299] Negotiation successful
0812 01:23:36.552582 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 01:23:36.553157 30099 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.0.188" port: 32789 } has no permanent_uuid. Determining permanent_uuid...
I20260812 01:23:36.553650 29858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:36.544594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46056 (local address 127.24.0.190:37259)
0812 01:23:36.544634 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:36.544636 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:36.544860 (+ 224us) server_negotiation.cc:407] Connection header received
0812 01:23:36.544934 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:36.544935 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:36.544946 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:36.544995 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:23:36.545605 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:36.546021 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:36.546732 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:36.546940 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:36.547004 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:36.549833 (+ 2829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:23:36.549834 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:23:36.549835 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:23:36.549842 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:23:36.552171 (+ 2329us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:23:36.552577 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:23:36.552579 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:23:36.552579 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:23:36.552635 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:23:36.552853 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:23:36.552854 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:23:36.552854 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:23:36.553015 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:23:36.553212 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:36.553466 (+ 254us) server_negotiation.cc:299] Negotiation successful
0812 01:23:36.553545 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 01:23:36.555797 29857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:36.547170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.188:32789 (local address 127.0.0.1:35860)
0812 01:23:36.547270 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0812 01:23:36.547281 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 01:23:36.547318 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:23:36.547939 (+ 621us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:23:36.547940 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:23:36.548129 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:23:36.548175 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:23:36.548177 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:23:36.549193 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:23:36.549194 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:23:36.549844 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:23:36.549846 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:23:36.550238 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:23:36.550240 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:23:36.550303 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:36.550750 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:23:36.550754 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:23:36.552277 (+ 1523us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:23:36.554611 (+ 2334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:23:36.554612 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:23:36.554612 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:23:36.554881 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:23:36.555133 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:23:36.555134 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:23:36.555135 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:23:36.555214 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:23:36.555472 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:23:36.555473 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:23:36.555515 (+ 42us) client_negotiation.cc:769] Sending connection context
0812 01:23:36.555611 (+ 96us) client_negotiation.cc:240] Negotiation successful
0812 01:23:36.555707 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84}
I20260812 01:23:36.555794 30104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:36.547730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35860 (local address 127.24.0.188:32789)
0812 01:23:36.547801 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:36.547803 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:36.547812 (+ 9us) server_negotiation.cc:407] Connection header received
0812 01:23:36.547832 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:36.547833 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:36.547844 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:36.548256 (+ 412us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:23:36.548290 (+ 34us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:36.549046 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:36.549934 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:36.550120 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:36.550234 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:36.552359 (+ 2125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:23:36.552360 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:23:36.552361 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:23:36.552367 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:23:36.554522 (+ 2155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:23:36.554989 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:23:36.554990 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:23:36.554990 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:23:36.555049 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:23:36.555272 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:23:36.555273 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:23:36.555273 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:23:36.555387 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:23:36.555467 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:36.555637 (+ 170us) server_negotiation.cc:299] Negotiation successful
0812 01:23:36.555712 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 01:23:36.557914 29854 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84f90d3c789a459a9b28b36b6ddf155b: Bootstrap starting.
Aug 12 01:23:36 dist-test-slave-t3q3 krb5kdc[29733](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497816, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.188@KRBTEST.COM for kudu/127.24.0.188@KRBTEST.COM
I20260812 01:23:36.558761 29854 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 84f90d3c789a459a9b28b36b6ddf155b: Neither blocks nor log segments found. Creating new log.
I20260812 01:23:36.559185 29854 log.cc:824] T 00000000000000000000000000000000 P 84f90d3c789a459a9b28b36b6ddf155b: Log is configured to *not* fsync() on all Append() calls
Aug 12 01:23:36 dist-test-slave-t3q3 krb5kdc[29733](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497816, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.0.189@KRBTEST.COM
I20260812 01:23:36.560312 29854 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84f90d3c789a459a9b28b36b6ddf155b: No bootstrap required, opened a new log
I20260812 01:23:36.561956 29854 raft_consensus.cc:348] T 00000000000000000000000000000000 P 84f90d3c789a459a9b28b36b6ddf155b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84f90d3c789a459a9b28b36b6ddf155b" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 37259 } } peers { permanent_uuid: "b35fca45a01942b386f5656876b7d5ce" member_type: VOTER last_known_addr { host: "127.24.0.189" port: 38941 } } peers { permanent_uuid: "4bcab980295242c9962ef9088dd6c5f9" member_type: VOTER last_known_addr { host: "127.24.0.188" port: 32789 } }
I20260812 01:23:36.562117 29854 raft_consensus.cc:729] T 00000000000000000000000000000000 P 84f90d3c789a459a9b28b36b6ddf155b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84f90d3c789a459a9b28b36b6ddf155b, State: Initialized, Role: FOLLOWER
I20260812 01:23:36.562219 29854 consensus_queue.cc:260] T 00000000000000000000000000000000 P 84f90d3c789a459a9b28b36b6ddf155b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84f90d3c789a459a9b28b36b6ddf155b" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 37259 } } peers { permanent_uuid: "b35fca45a01942b386f5656876b7d5ce" member_type: VOTER last_known_addr { host: "127.24.0.189" port: 38941 } } peers { permanent_uuid: "4bcab980295242c9962ef9088dd6c5f9" member_type: VOTER last_known_addr { host: "127.24.0.188" port: 32789 } }
I20260812 01:23:36.562610 29854 sys_catalog.cc:564] T 00000000000000000000000000000000 P 84f90d3c789a459a9b28b36b6ddf155b [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:23:36.562793 29985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:36.554520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34548 (local address 127.24.0.189:38941)
0812 01:23:36.554555 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:36.554557 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:36.554597 (+ 40us) server_negotiation.cc:407] Connection header received
0812 01:23:36.554656 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:36.554657 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:36.554667 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:36.554711 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:23:36.555189 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:36.555847 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:36.556784 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:36.557088 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:36.557162 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:36.559851 (+ 2689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:23:36.559852 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:23:36.559853 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:23:36.559859 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:23:36.561512 (+ 1653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:23:36.561910 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:23:36.561911 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:23:36.561912 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:23:36.561999 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:23:36.562235 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:23:36.562236 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:23:36.562236 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:23:36.562382 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:23:36.562549 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:36.562587 (+ 38us) server_negotiation.cc:299] Negotiation successful
0812 01:23:36.562662 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260812 01:23:36.563009 30110 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84f90d3c789a459a9b28b36b6ddf155b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84f90d3c789a459a9b28b36b6ddf155b" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 37259 } } peers { permanent_uuid: "b35fca45a01942b386f5656876b7d5ce" member_type: VOTER last_known_addr { host: "127.24.0.189" port: 38941 } } peers { permanent_uuid: "4bcab980295242c9962ef9088dd6c5f9" member_type: VOTER last_known_addr { host: "127.24.0.188" port: 32789 } } }
I20260812 01:23:36.563138 30110 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84f90d3c789a459a9b28b36b6ddf155b [sys.catalog]: This master's current role is: FOLLOWER
I20260812 01:23:36.564388 24578 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 01:23:36.565339 29815 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "84f90d3c789a459a9b28b36b6ddf155b"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46056
I20260812 01:23:36.565459 29815 raft_consensus.cc:482] T 00000000000000000000000000000000 P 84f90d3c789a459a9b28b36b6ddf155b [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 01:23:36.565500 29815 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 84f90d3c789a459a9b28b36b6ddf155b [term 0 FOLLOWER]: Advancing to term 1
W20260812 01:23:36.565507 30130 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 84f90d3c789a459a9b28b36b6ddf155b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 01:23:36.565569 30130 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 01:23:36.566059 30106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:36.553403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.188:32789 (local address 127.0.0.1:35872)
0812 01:23:36.553633 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0812 01:23:36.553643 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 01:23:36.553895 (+ 252us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:23:36.554086 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:23:36.554087 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:23:36.554303 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:23:36.554337 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:23:36.554339 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:23:36.555801 (+ 1462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:23:36.555801 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:23:36.556695 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:23:36.556698 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:23:36.557216 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:23:36.557217 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:23:36.557261 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:36.557836 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:23:36.557841 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:23:36.559559 (+ 1718us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:23:36.564713 (+ 5154us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:23:36.564714 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:23:36.564715 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:23:36.564948 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:23:36.565151 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:23:36.565151 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:23:36.565152 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:23:36.565207 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:23:36.565665 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:23:36.565666 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:23:36.565709 (+ 43us) client_negotiation.cc:769] Sending connection context
0812 01:23:36.565818 (+ 109us) client_negotiation.cc:240] Negotiation successful
0812 01:23:36.565925 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214}
I20260812 01:23:36.566170 30107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:36.553473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35872 (local address 127.24.0.188:32789)
0812 01:23:36.553760 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:36.553762 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:36.553771 (+ 9us) server_negotiation.cc:407] Connection header received
0812 01:23:36.553981 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:36.553982 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:36.553994 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:36.554052 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:23:36.554416 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:36.555721 (+ 1305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:36.556814 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:36.557120 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:36.557172 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:36.559664 (+ 2492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:23:36.559666 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:23:36.559666 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:23:36.559674 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:23:36.564551 (+ 4877us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:23:36.565028 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:23:36.565029 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:23:36.565030 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:23:36.565087 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:23:36.565269 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:23:36.565270 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:23:36.565270 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:23:36.565433 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:23:36.565494 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:36.565844 (+ 350us) server_negotiation.cc:299] Negotiation successful
0812 01:23:36.565979 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":68,"server-negotiator.queue_time_us":263,"thread_start_us":103,"threads_started":1}
I20260812 01:23:36.566484 29815 raft_consensus.cc:504] T 00000000000000000000000000000000 P 84f90d3c789a459a9b28b36b6ddf155b [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84f90d3c789a459a9b28b36b6ddf155b" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 37259 } } peers { permanent_uuid: "b35fca45a01942b386f5656876b7d5ce" member_type: VOTER last_known_addr { host: "127.24.0.189" port: 38941 } } peers { permanent_uuid: "4bcab980295242c9962ef9088dd6c5f9" member_type: VOTER last_known_addr { host: "127.24.0.188" port: 32789 } }
I20260812 01:23:36.566752 29815 leader_election.cc:288] T 00000000000000000000000000000000 P 84f90d3c789a459a9b28b36b6ddf155b [CANDIDATE]: Term 1 election: Requested vote from peers b35fca45a01942b386f5656876b7d5ce (127.24.0.189:38941), 4bcab980295242c9962ef9088dd6c5f9 (127.24.0.188:32789)
I20260812 01:23:36.567941 30099 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4bcab980295242c9962ef9088dd6c5f9: Bootstrap starting.
I20260812 01:23:36.568761 30099 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4bcab980295242c9962ef9088dd6c5f9: Neither blocks nor log segments found. Creating new log.
I20260812 01:23:36.569072 30099 log.cc:824] T 00000000000000000000000000000000 P 4bcab980295242c9962ef9088dd6c5f9: Log is configured to *not* fsync() on all Append() calls
I20260812 01:23:36.566039 29756 master_runner.cc:186] Error getting master registration for 127.24.0.189:38941: OK, instance_id { permanent_uuid: "b35fca45a01942b386f5656876b7d5ce" instance_seqno: 1786497816362942 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 01:23:36.569147 29756 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 01:23:36.570101 30099 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4bcab980295242c9962ef9088dd6c5f9: No bootstrap required, opened a new log
I20260812 01:23:36.572671 30099 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4bcab980295242c9962ef9088dd6c5f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84f90d3c789a459a9b28b36b6ddf155b" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 37259 } } peers { permanent_uuid: "b35fca45a01942b386f5656876b7d5ce" member_type: VOTER last_known_addr { host: "127.24.0.189" port: 38941 } } peers { permanent_uuid: "4bcab980295242c9962ef9088dd6c5f9" member_type: VOTER last_known_addr { host: "127.24.0.188" port: 32789 } }
I20260812 01:23:36.572834 30099 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4bcab980295242c9962ef9088dd6c5f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bcab980295242c9962ef9088dd6c5f9, State: Initialized, Role: FOLLOWER
I20260812 01:23:36.572932 30099 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4bcab980295242c9962ef9088dd6c5f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84f90d3c789a459a9b28b36b6ddf155b" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 37259 } } peers { permanent_uuid: "b35fca45a01942b386f5656876b7d5ce" member_type: VOTER last_known_addr { host: "127.24.0.189" port: 38941 } } peers { permanent_uuid: "4bcab980295242c9962ef9088dd6c5f9" member_type: VOTER last_known_addr { host: "127.24.0.188" port: 32789 } }
I20260812 01:23:36.573273 30148 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4bcab980295242c9962ef9088dd6c5f9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84f90d3c789a459a9b28b36b6ddf155b" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 37259 } } peers { permanent_uuid: "b35fca45a01942b386f5656876b7d5ce" member_type: VOTER last_known_addr { host: "127.24.0.189" port: 38941 } } peers { permanent_uuid: "4bcab980295242c9962ef9088dd6c5f9" member_type: VOTER last_known_addr { host: "127.24.0.188" port: 32789 } } }
I20260812 01:23:36.573324 30059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "84f90d3c789a459a9b28b36b6ddf155b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "4bcab980295242c9962ef9088dd6c5f9"
I20260812 01:23:36.573293 30099 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4bcab980295242c9962ef9088dd6c5f9 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:23:36.573642 30059 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4bcab980295242c9962ef9088dd6c5f9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:23:36.575165 30059 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4bcab980295242c9962ef9088dd6c5f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84f90d3c789a459a9b28b36b6ddf155b in term 1.
I20260812 01:23:36.575460 29780 leader_election.cc:302] T 00000000000000000000000000000000 P 84f90d3c789a459a9b28b36b6ddf155b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bcab980295242c9962ef9088dd6c5f9, 84f90d3c789a459a9b28b36b6ddf155b; no voters:
I20260812 01:23:36.573572 30148 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4bcab980295242c9962ef9088dd6c5f9 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 01:23:36.577445 30110 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 84f90d3c789a459a9b28b36b6ddf155b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:23:36.577637 30110 raft_consensus.cc:686] T 00000000000000000000000000000000 P 84f90d3c789a459a9b28b36b6ddf155b [term 1 LEADER]: Becoming Leader. State: Replica: 84f90d3c789a459a9b28b36b6ddf155b, State: Running, Role: LEADER
I20260812 01:23:36.578162 30110 consensus_queue.cc:237] T 00000000000000000000000000000000 P 84f90d3c789a459a9b28b36b6ddf155b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84f90d3c789a459a9b28b36b6ddf155b" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 37259 } } peers { permanent_uuid: "b35fca45a01942b386f5656876b7d5ce" member_type: VOTER last_known_addr { host: "127.24.0.189" port: 38941 } } peers { permanent_uuid: "4bcab980295242c9962ef9088dd6c5f9" member_type: VOTER last_known_addr { host: "127.24.0.188" port: 32789 } }
I20260812 01:23:36.579728 30163 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84f90d3c789a459a9b28b36b6ddf155b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 84f90d3c789a459a9b28b36b6ddf155b. Latest consensus state: current_term: 1 leader_uuid: "84f90d3c789a459a9b28b36b6ddf155b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84f90d3c789a459a9b28b36b6ddf155b" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 37259 } } peers { permanent_uuid: "b35fca45a01942b386f5656876b7d5ce" member_type: VOTER last_known_addr { host: "127.24.0.189" port: 38941 } } peers { permanent_uuid: "4bcab980295242c9962ef9088dd6c5f9" member_type: VOTER last_known_addr { host: "127.24.0.188" port: 32789 } } }
I20260812 01:23:36.579993 30163 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84f90d3c789a459a9b28b36b6ddf155b [sys.catalog]: This master's current role is: LEADER
I20260812 01:23:36.580565 30107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:36.569435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35876 (local address 127.24.0.188:32789)
0812 01:23:36.569468 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:36.569469 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:36.572250 (+ 2781us) server_negotiation.cc:407] Connection header received
0812 01:23:36.572308 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:36.572309 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:36.572323 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:36.572356 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:23:36.573097 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:36.573800 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:36.574731 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:36.575061 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:36.575103 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:36.577437 (+ 2334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:23:36.577438 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:23:36.577439 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:23:36.577446 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:23:36.579210 (+ 1764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:23:36.579618 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:23:36.579619 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:23:36.579620 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:23:36.579701 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:23:36.579930 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:23:36.579931 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:23:36.579931 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:23:36.580091 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:23:36.580175 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:36.580405 (+ 230us) server_negotiation.cc:299] Negotiation successful
0812 01:23:36.580484 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 01:23:36.578436 29991 master_runner.cc:186] Error getting master registration for 127.24.0.189:38941: OK, instance_id { permanent_uuid: "b35fca45a01942b386f5656876b7d5ce" instance_seqno: 1786497816362942 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 01:23:36.580703 29991 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 01:23:36.583328 29985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:36.568293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34556 (local address 127.24.0.189:38941)
0812 01:23:36.569238 (+ 945us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:36.569240 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:36.569250 (+ 10us) server_negotiation.cc:407] Connection header received
0812 01:23:36.569271 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:36.569272 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:36.569281 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:36.569347 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:23:36.572633 (+ 3286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:36.573384 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:36.574346 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:36.574669 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:36.574721 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:36.576434 (+ 1713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:23:36.576435 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:23:36.576435 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:23:36.576443 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:23:36.581811 (+ 5368us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:23:36.582289 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:23:36.582290 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:23:36.582290 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:23:36.582356 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:23:36.582567 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:23:36.582568 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:23:36.582568 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:23:36.582716 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:23:36.582804 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:36.583153 (+ 349us) server_negotiation.cc:299] Negotiation successful
0812 01:23:36.583235 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":920}
I20260812 01:23:36.583683 29858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:36.568232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46064 (local address 127.24.0.190:37259)
0812 01:23:36.568273 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:36.568275 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:36.569709 (+ 1434us) server_negotiation.cc:407] Connection header received
0812 01:23:36.569804 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:36.569804 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:36.569815 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:36.569845 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:23:36.570499 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:36.571205 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:36.572360 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:36.572663 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:36.572727 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:36.577566 (+ 4839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:23:36.577567 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:23:36.577567 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:23:36.577574 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:23:36.582184 (+ 4610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:23:36.582587 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:23:36.582588 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:23:36.582588 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:23:36.582648 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:23:36.582866 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:23:36.582866 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:23:36.582867 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:23:36.583000 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:23:36.583110 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:36.583500 (+ 390us) server_negotiation.cc:299] Negotiation successful
0812 01:23:36.583585 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
W20260812 01:23:36.580986 30162 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4bcab980295242c9962ef9088dd6c5f9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 01:23:36.583895 30162 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 01:23:36.584040 30166 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:23:36.584304 30166 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:23:36.585927 30059 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4bcab980295242c9962ef9088dd6c5f9 [term 1 FOLLOWER]: Refusing update from remote peer 84f90d3c789a459a9b28b36b6ddf155b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:23:36.586243 30110 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 84f90d3c789a459a9b28b36b6ddf155b [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcab980295242c9962ef9088dd6c5f9" member_type: VOTER last_known_addr { host: "127.24.0.188" port: 32789 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:23:36.587464 30148 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4bcab980295242c9962ef9088dd6c5f9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 84f90d3c789a459a9b28b36b6ddf155b. Latest consensus state: current_term: 1 leader_uuid: "84f90d3c789a459a9b28b36b6ddf155b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84f90d3c789a459a9b28b36b6ddf155b" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 37259 } } peers { permanent_uuid: "b35fca45a01942b386f5656876b7d5ce" member_type: VOTER last_known_addr { host: "127.24.0.189" port: 38941 } } peers { permanent_uuid: "4bcab980295242c9962ef9088dd6c5f9" member_type: VOTER last_known_addr { host: "127.24.0.188" port: 32789 } } }
I20260812 01:23:36.587558 30148 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4bcab980295242c9962ef9088dd6c5f9 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 01:23:36.594242 30163 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84f90d3c789a459a9b28b36b6ddf155b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "84f90d3c789a459a9b28b36b6ddf155b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84f90d3c789a459a9b28b36b6ddf155b" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 37259 } } peers { permanent_uuid: "b35fca45a01942b386f5656876b7d5ce" member_type: VOTER last_known_addr { host: "127.24.0.189" port: 38941 } } peers { permanent_uuid: "4bcab980295242c9962ef9088dd6c5f9" member_type: VOTER last_known_addr { host: "127.24.0.188" port: 32789 } } }
I20260812 01:23:36.594342 30163 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84f90d3c789a459a9b28b36b6ddf155b [sys.catalog]: This master's current role is: LEADER
I20260812 01:23:36.594784 30148 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4bcab980295242c9962ef9088dd6c5f9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "84f90d3c789a459a9b28b36b6ddf155b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84f90d3c789a459a9b28b36b6ddf155b" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 37259 } } peers { permanent_uuid: "b35fca45a01942b386f5656876b7d5ce" member_type: VOTER last_known_addr { host: "127.24.0.189" port: 38941 } } peers { permanent_uuid: "4bcab980295242c9962ef9088dd6c5f9" member_type: VOTER last_known_addr { host: "127.24.0.188" port: 32789 } } }
I20260812 01:23:36.594863 30148 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4bcab980295242c9962ef9088dd6c5f9 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 01:23:36.595057 30169 mvcc.cc:205] Tried to move back new op lower bound from 7317495056733368320 to 7317495056706965504. Current Snapshot: MvccSnapshot[applied={T|T < 7317495056733368320 or (T in {7317495056733368320})}]
I20260812 01:23:36.595170 30170 mvcc.cc:205] Tried to move back new op lower bound from 7317495056733368320 to 7317495056706965504. Current Snapshot: MvccSnapshot[applied={T|T < 7317495056733368320}]
I20260812 01:23:36.595291 30166 catalog_manager.cc:1347] Generated new cluster ID: 9681a553dbb84e90bf55ec27bb623966
I20260812 01:23:36.595338 30166 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 01:23:36.598110 29981 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.0.188" port: 32789 } attempt: 3
Aug 12 01:23:36 dist-test-slave-t3q3 krb5kdc[29733](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497816, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.0.189@KRBTEST.COM for kudu/127.24.0.188@KRBTEST.COM
I20260812 01:23:36.605665 30166 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:23:36.606719 29983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:36.598389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.0.188:32789 (local address 127.0.0.1:35878)
0812 01:23:36.598459 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0812 01:23:36.598474 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 01:23:36.598526 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:23:36.598714 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:23:36.598715 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:23:36.598922 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:23:36.598992 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:23:36.598994 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:23:36.599934 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:23:36.599935 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:23:36.600733 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:23:36.600734 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:23:36.601189 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:23:36.601190 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:23:36.601246 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:36.601731 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:23:36.601737 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:23:36.603278 (+ 1541us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:23:36.605553 (+ 2275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:23:36.605554 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:23:36.605554 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:23:36.605799 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:23:36.606034 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:23:36.606035 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:23:36.606036 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:23:36.606101 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:23:36.606407 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:23:36.606408 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:23:36.606455 (+ 47us) client_negotiation.cc:769] Sending connection context
0812 01:23:36.606517 (+ 62us) client_negotiation.cc:240] Negotiation successful
0812 01:23:36.606623 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260812 01:23:36.607399 30166 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:23:36.607108 30107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:36.598550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35878 (local address 127.24.0.188:32789)
0812 01:23:36.598594 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:36.598596 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:36.598603 (+ 7us) server_negotiation.cc:407] Connection header received
0812 01:23:36.598624 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:36.598625 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:36.598635 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:36.598710 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:23:36.599080 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:36.599846 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:36.600814 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:36.601116 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:36.601183 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:36.603392 (+ 2209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:23:36.603394 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:23:36.603395 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:23:36.603403 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:23:36.605475 (+ 2072us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:23:36.605892 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:23:36.605893 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:23:36.605894 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:23:36.605960 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:23:36.606182 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:23:36.606182 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:23:36.606183 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:23:36.606316 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:23:36.606380 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:36.606937 (+ 557us) server_negotiation.cc:299] Negotiation successful
0812 01:23:36.607021 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 01:23:36.609136 29981 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b35fca45a01942b386f5656876b7d5ce: Bootstrap starting.
I20260812 01:23:36.609844 29981 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b35fca45a01942b386f5656876b7d5ce: Neither blocks nor log segments found. Creating new log.
I20260812 01:23:36.610128 29981 log.cc:824] T 00000000000000000000000000000000 P b35fca45a01942b386f5656876b7d5ce: Log is configured to *not* fsync() on all Append() calls
I20260812 01:23:36.611183 29981 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b35fca45a01942b386f5656876b7d5ce: No bootstrap required, opened a new log
I20260812 01:23:36.614552 29981 raft_consensus.cc:348] T 00000000000000000000000000000000 P b35fca45a01942b386f5656876b7d5ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84f90d3c789a459a9b28b36b6ddf155b" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 37259 } } peers { permanent_uuid: "b35fca45a01942b386f5656876b7d5ce" member_type: VOTER last_known_addr { host: "127.24.0.189" port: 38941 } } peers { permanent_uuid: "4bcab980295242c9962ef9088dd6c5f9" member_type: VOTER last_known_addr { host: "127.24.0.188" port: 32789 } }
I20260812 01:23:36.616142 29981 raft_consensus.cc:729] T 00000000000000000000000000000000 P b35fca45a01942b386f5656876b7d5ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b35fca45a01942b386f5656876b7d5ce, State: Initialized, Role: FOLLOWER
I20260812 01:23:36.616284 29981 consensus_queue.cc:260] T 00000000000000000000000000000000 P b35fca45a01942b386f5656876b7d5ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84f90d3c789a459a9b28b36b6ddf155b" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 37259 } } peers { permanent_uuid: "b35fca45a01942b386f5656876b7d5ce" member_type: VOTER last_known_addr { host: "127.24.0.189" port: 38941 } } peers { permanent_uuid: "4bcab980295242c9962ef9088dd6c5f9" member_type: VOTER last_known_addr { host: "127.24.0.188" port: 32789 } }
I20260812 01:23:36.616822 30166 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 84f90d3c789a459a9b28b36b6ddf155b: Generated new TSK 0
I20260812 01:23:36.616742 30177 sys_catalog.cc:455] T 00000000000000000000000000000000 P b35fca45a01942b386f5656876b7d5ce [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84f90d3c789a459a9b28b36b6ddf155b" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 37259 } } peers { permanent_uuid: "b35fca45a01942b386f5656876b7d5ce" member_type: VOTER last_known_addr { host: "127.24.0.189" port: 38941 } } peers { permanent_uuid: "4bcab980295242c9962ef9088dd6c5f9" member_type: VOTER last_known_addr { host: "127.24.0.188" port: 32789 } } }
I20260812 01:23:36.616966 30177 sys_catalog.cc:458] T 00000000000000000000000000000000 P b35fca45a01942b386f5656876b7d5ce [sys.catalog]: This master's current role is: FOLLOWER
I20260812 01:23:36.616786 29938 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "84f90d3c789a459a9b28b36b6ddf155b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "b35fca45a01942b386f5656876b7d5ce"
I20260812 01:23:36.617163 30166 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 01:23:36.616859 29935 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b35fca45a01942b386f5656876b7d5ce [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:23:36.618178 29935 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b35fca45a01942b386f5656876b7d5ce [term 1 FOLLOWER]: Refusing update from remote peer 84f90d3c789a459a9b28b36b6ddf155b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:23:36.618487 30163 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 84f90d3c789a459a9b28b36b6ddf155b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b35fca45a01942b386f5656876b7d5ce" member_type: VOTER last_known_addr { host: "127.24.0.189" port: 38941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:23:36.619020 29938 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b35fca45a01942b386f5656876b7d5ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84f90d3c789a459a9b28b36b6ddf155b in term 1.
I20260812 01:23:36.616782 29981 sys_catalog.cc:564] T 00000000000000000000000000000000 P b35fca45a01942b386f5656876b7d5ce [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:23:36.622159 30177 sys_catalog.cc:455] T 00000000000000000000000000000000 P b35fca45a01942b386f5656876b7d5ce [sys.catalog]: SysCatalogTable state changed. Reason: New leader 84f90d3c789a459a9b28b36b6ddf155b. Latest consensus state: current_term: 1 leader_uuid: "84f90d3c789a459a9b28b36b6ddf155b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84f90d3c789a459a9b28b36b6ddf155b" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 37259 } } peers { permanent_uuid: "b35fca45a01942b386f5656876b7d5ce" member_type: VOTER last_known_addr { host: "127.24.0.189" port: 38941 } } peers { permanent_uuid: "4bcab980295242c9962ef9088dd6c5f9" member_type: VOTER last_known_addr { host: "127.24.0.188" port: 32789 } } }
I20260812 01:23:36.622260 30177 sys_catalog.cc:458] T 00000000000000000000000000000000 P b35fca45a01942b386f5656876b7d5ce [sys.catalog]: This master's current role is: FOLLOWER
I20260812 01:23:36.622298 30177 sys_catalog.cc:455] T 00000000000000000000000000000000 P b35fca45a01942b386f5656876b7d5ce [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "84f90d3c789a459a9b28b36b6ddf155b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84f90d3c789a459a9b28b36b6ddf155b" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 37259 } } peers { permanent_uuid: "b35fca45a01942b386f5656876b7d5ce" member_type: VOTER last_known_addr { host: "127.24.0.189" port: 38941 } } peers { permanent_uuid: "4bcab980295242c9962ef9088dd6c5f9" member_type: VOTER last_known_addr { host: "127.24.0.188" port: 32789 } } }
I20260812 01:23:36.622345 30177 sys_catalog.cc:458] T 00000000000000000000000000000000 P b35fca45a01942b386f5656876b7d5ce [sys.catalog]: This master's current role is: FOLLOWER
I20260812 01:23:36.622731 30206 catalog_manager.cc:1259] Loaded cluster ID: 9681a553dbb84e90bf55ec27bb623966
I20260812 01:23:36.622815 30206 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b35fca45a01942b386f5656876b7d5ce: loading cluster ID for follower catalog manager: success
I20260812 01:23:36.624029 30206 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b35fca45a01942b386f5656876b7d5ce: acquiring CA information for follower catalog manager: success
I20260812 01:23:36.624485 30206 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b35fca45a01942b386f5656876b7d5ce: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260812 01:23:36.651542 29985 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:23:36.649842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34564 (local address 127.24.0.189:38941)
0812 01:23:36.649891 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:36.649893 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:36.650838 (+ 945us) server_negotiation.cc:407] Connection header received
0812 01:23:36.650955 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:36.650956 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:36.650967 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:36.650999 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:23:36.651470 (+ 471us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34564: BlockingRecv error: recv got EOF from 127.0.0.1:34564 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260812 01:23:36.652340 29858 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:23:36.650271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46066 (local address 127.24.0.190:37259)
0812 01:23:36.650315 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:36.650316 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:36.651788 (+ 1472us) server_negotiation.cc:407] Connection header received
0812 01:23:36.651888 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:36.651889 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:36.651902 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:36.651932 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:23:36.652276 (+ 344us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46066: BlockingRecv error: recv got EOF from 127.0.0.1:46066 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260812 01:23:36.653110 30107 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:23:36.650572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35888 (local address 127.24.0.188:32789)
0812 01:23:36.650610 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:36.650612 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:36.652546 (+ 1934us) server_negotiation.cc:407] Connection header received
0812 01:23:36.652673 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:36.652674 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:36.652685 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:36.652715 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:23:36.653046 (+ 331us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35888: BlockingRecv error: recv got EOF from 127.0.0.1:35888 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 01:23:36.659935 29985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:36.656369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34576 (local address 127.24.0.189:38941)
0812 01:23:36.656414 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:36.656416 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:36.656590 (+ 174us) server_negotiation.cc:407] Connection header received
0812 01:23:36.656697 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:36.656698 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:36.656741 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:36.656789 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:23:36.656998 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:36.657790 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:36.658963 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:36.659276 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:36.659316 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:36.659541 (+ 225us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:23:36.659678 (+ 137us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:23:36.659731 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:36.659847 (+ 116us) server_negotiation.cc:299] Negotiation successful
0812 01:23:36.659866 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
W20260812 01:23:36.663143 30107 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:23:36.662334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35900 (local address 127.24.0.188:32789)
0812 01:23:36.662371 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:36.662373 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:36.662608 (+ 235us) server_negotiation.cc:407] Connection header received
0812 01:23:36.662719 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:36.662720 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:36.662740 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:36.662770 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:23:36.663094 (+ 324us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35900: BlockingRecv error: recv got EOF from 127.0.0.1:35900 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 01:23:36.668568 29985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:36.665367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34582 (local address 127.24.0.189:38941)
0812 01:23:36.665398 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:36.665400 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:36.665628 (+ 228us) server_negotiation.cc:407] Connection header received
0812 01:23:36.665720 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:36.665721 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:36.665729 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:36.665757 (+ 28us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:23:36.665915 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:36.666627 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:36.667757 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:36.668044 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:36.668106 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:36.668318 (+ 212us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:23:36.668352 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:23:36.668380 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:36.668483 (+ 103us) server_negotiation.cc:299] Negotiation successful
0812 01:23:36.668501 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
W20260812 01:23:36.671550 30107 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:23:36.670825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35914 (local address 127.24.0.188:32789)
0812 01:23:36.670859 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:36.670861 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:36.671049 (+ 188us) server_negotiation.cc:407] Connection header received
0812 01:23:36.671145 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:36.671147 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:36.671168 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:36.671198 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:23:36.671497 (+ 299us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35914: BlockingRecv error: recv got EOF from 127.0.0.1:35914 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 01:23:36.678026 29985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:36.674746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34586 (local address 127.24.0.189:38941)
0812 01:23:36.674778 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:36.674780 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:36.675000 (+ 220us) server_negotiation.cc:407] Connection header received
0812 01:23:36.675096 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:36.675097 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:36.675105 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:36.675133 (+ 28us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:23:36.675343 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:36.676199 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:36.677285 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:36.677552 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:36.677589 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:36.677775 (+ 186us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:23:36.677807 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:23:36.677834 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:36.677938 (+ 104us) server_negotiation.cc:299] Negotiation successful
0812 01:23:36.677954 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
W20260812 01:23:36.681108 30107 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:23:36.680367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35930 (local address 127.24.0.188:32789)
0812 01:23:36.680406 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:36.680408 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:36.680597 (+ 189us) server_negotiation.cc:407] Connection header received
0812 01:23:36.680694 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:36.680695 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:36.680706 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:36.680736 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:23:36.681058 (+ 322us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35930: BlockingRecv error: recv got EOF from 127.0.0.1:35930 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 01:23:36.689677 29985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:36.686304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34596 (local address 127.24.0.189:38941)
0812 01:23:36.686343 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:36.686345 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:36.686602 (+ 257us) server_negotiation.cc:407] Connection header received
0812 01:23:36.686706 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:36.686707 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:36.686717 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:36.686747 (+ 30us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:23:36.686948 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:36.687707 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:36.688854 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:36.689150 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:36.689204 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:36.689411 (+ 207us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:23:36.689445 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:23:36.689473 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:36.689578 (+ 105us) server_negotiation.cc:299] Negotiation successful
0812 01:23:36.689598 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
W20260812 01:23:36.692929 30107 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:23:36.692124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35938 (local address 127.24.0.188:32789)
0812 01:23:36.692176 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:36.692178 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:36.692376 (+ 198us) server_negotiation.cc:407] Connection header received
0812 01:23:36.692485 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:36.692486 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:36.692497 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:36.692527 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:23:36.692871 (+ 344us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35938: BlockingRecv error: recv got EOF from 127.0.0.1:35938 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 01:23:36.705572 29985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:36.702213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34606 (local address 127.24.0.189:38941)
0812 01:23:36.702256 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:36.702258 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:36.702537 (+ 279us) server_negotiation.cc:407] Connection header received
0812 01:23:36.702640 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:36.702641 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:36.702652 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:36.702683 (+ 31us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:23:36.702856 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:36.703613 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:36.704782 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:36.705072 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:36.705110 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:36.705314 (+ 204us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:23:36.705348 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:23:36.705375 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:36.705475 (+ 100us) server_negotiation.cc:299] Negotiation successful
0812 01:23:36.705493 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
W20260812 01:23:36.708715 30107 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:23:36.707875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35940 (local address 127.24.0.188:32789)
0812 01:23:36.707918 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:36.707920 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:36.708149 (+ 229us) server_negotiation.cc:407] Connection header received
0812 01:23:36.708265 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:36.708266 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:36.708278 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:36.708308 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:23:36.708662 (+ 354us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35940: BlockingRecv error: recv got EOF from 127.0.0.1:35940 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 01:23:36.737905 29985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:36.726181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34608 (local address 127.24.0.189:38941)
0812 01:23:36.726231 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:36.726233 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:36.726581 (+ 348us) server_negotiation.cc:407] Connection header received
0812 01:23:36.726702 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:36.726704 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:36.726715 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:36.726747 (+ 32us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:23:36.726960 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:36.727715 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:36.736966 (+ 9251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:36.737283 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:36.737367 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:36.737612 (+ 245us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:23:36.737652 (+ 40us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:23:36.737682 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:36.737794 (+ 112us) server_negotiation.cc:299] Negotiation successful
0812 01:23:36.737819 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
W20260812 01:23:36.741557 30107 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:23:36.740631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35956 (local address 127.24.0.188:32789)
0812 01:23:36.740678 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:36.740679 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:36.740920 (+ 241us) server_negotiation.cc:407] Connection header received
0812 01:23:36.741028 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:36.741028 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:36.741041 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:36.741074 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:23:36.741497 (+ 423us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35956: BlockingRecv error: recv got EOF from 127.0.0.1:35956 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 01:23:36.778602 29985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:36.775042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34610 (local address 127.24.0.189:38941)
0812 01:23:36.775090 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:36.775092 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:36.775487 (+ 395us) server_negotiation.cc:407] Connection header received
0812 01:23:36.775596 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:36.775598 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:36.775609 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:36.775638 (+ 29us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:23:36.775832 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:36.776627 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:36.777768 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:36.778060 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:36.778103 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:36.778325 (+ 222us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:23:36.778358 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:23:36.778389 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:36.778501 (+ 112us) server_negotiation.cc:299] Negotiation successful
0812 01:23:36.778522 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
W20260812 01:23:36.781694 30107 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:23:36.780945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35962 (local address 127.24.0.188:32789)
0812 01:23:36.780984 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:36.780986 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:36.781172 (+ 186us) server_negotiation.cc:407] Connection header received
0812 01:23:36.781267 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:36.781268 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:36.781280 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:36.781308 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:23:36.781642 (+ 334us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35962: BlockingRecv error: recv got EOF from 127.0.0.1:35962 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 01:23:36.849952 29985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:36.847198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34612 (local address 127.24.0.189:38941)
0812 01:23:36.847277 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:36.847279 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:36.847412 (+ 133us) server_negotiation.cc:407] Connection header received
0812 01:23:36.847494 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:36.847495 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:36.847507 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:36.847589 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:23:36.847751 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:36.848390 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:36.849211 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:36.849425 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:36.849510 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:36.849685 (+ 175us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:23:36.849720 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:23:36.849769 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:36.849845 (+ 76us) server_negotiation.cc:299] Negotiation successful
0812 01:23:36.849863 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
W20260812 01:23:36.852471 30107 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:23:36.851791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35964 (local address 127.24.0.188:32789)
0812 01:23:36.851872 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:36.851874 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:36.851886 (+ 12us) server_negotiation.cc:407] Connection header received
0812 01:23:36.851946 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:36.851947 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:36.851957 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:36.852005 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:23:36.852423 (+ 418us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35964: BlockingRecv error: recv got EOF from 127.0.0.1:35964 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 01:23:36.984131 29985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:36.981807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34618 (local address 127.24.0.189:38941)
0812 01:23:36.981889 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:36.981892 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:36.981973 (+ 81us) server_negotiation.cc:407] Connection header received
0812 01:23:36.982036 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:36.982037 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:36.982047 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:36.982103 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:23:36.982240 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:36.982719 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:36.983482 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:36.983658 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:36.983709 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:36.983855 (+ 146us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:23:36.983883 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:23:36.983921 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:36.983981 (+ 60us) server_negotiation.cc:299] Negotiation successful
0812 01:23:36.983996 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
W20260812 01:23:36.986482 30107 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:23:36.985963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35966 (local address 127.24.0.188:32789)
0812 01:23:36.986014 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:36.986016 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:36.986024 (+ 8us) server_negotiation.cc:407] Connection header received
0812 01:23:36.986064 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:36.986065 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:36.986075 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:36.986121 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:23:36.986437 (+ 316us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35966: BlockingRecv error: recv got EOF from 127.0.0.1:35966 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 01:23:37.179853 30372 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84f90d3c789a459a9b28b36b6ddf155b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "84f90d3c789a459a9b28b36b6ddf155b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84f90d3c789a459a9b28b36b6ddf155b" member_type: VOTER last_known_addr { host: "127.24.0.190" port: 37259 } } peers { permanent_uuid: "b35fca45a01942b386f5656876b7d5ce" member_type: VOTER last_known_addr { host: "127.24.0.189" port: 38941 } } peers { permanent_uuid: "4bcab980295242c9962ef9088dd6c5f9" member_type: VOTER last_known_addr { host: "127.24.0.188" port: 32789 } } }
I20260812 01:23:37.179997 30372 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84f90d3c789a459a9b28b36b6ddf155b [sys.catalog]: This master's current role is: LEADER
I20260812 01:23:37.246389 29985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:37.243758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34620 (local address 127.24.0.189:38941)
0812 01:23:37.243811 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:37.243814 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:37.243920 (+ 106us) server_negotiation.cc:407] Connection header received
0812 01:23:37.244002 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:37.244003 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:37.244013 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:37.244087 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:23:37.244287 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:37.244783 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:37.245640 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:37.245836 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:37.245896 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:37.246124 (+ 228us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:23:37.246159 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:23:37.246215 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:37.246306 (+ 91us) server_negotiation.cc:299] Negotiation successful
0812 01:23:37.246325 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
W20260812 01:23:37.248912 30107 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:23:37.248396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35976 (local address 127.24.0.188:32789)
0812 01:23:37.248449 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:37.248451 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:37.248459 (+ 8us) server_negotiation.cc:407] Connection header received
0812 01:23:37.248481 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:37.248482 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:37.248492 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:37.248538 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:23:37.248860 (+ 322us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35976: BlockingRecv error: recv got EOF from 127.0.0.1:35976 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 01:23:37.584421 30162 catalog_manager.cc:1259] Loaded cluster ID: 9681a553dbb84e90bf55ec27bb623966
I20260812 01:23:37.584507 30162 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 4bcab980295242c9962ef9088dd6c5f9: loading cluster ID for follower catalog manager: success
I20260812 01:23:37.585394 30162 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 4bcab980295242c9962ef9088dd6c5f9: acquiring CA information for follower catalog manager: success
I20260812 01:23:37.585762 30162 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 4bcab980295242c9962ef9088dd6c5f9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 01:23:37.764885 30403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:37.762134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34634 (local address 127.24.0.189:38941)
0812 01:23:37.762278 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:37.762281 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:37.762348 (+ 67us) server_negotiation.cc:407] Connection header received
0812 01:23:37.762421 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:37.762422 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:37.762435 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:37.762525 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:23:37.762686 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:37.763278 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:37.764109 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:37.764339 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:37.764427 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:37.764591 (+ 164us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:23:37.764630 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:23:37.764699 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:37.764759 (+ 60us) server_negotiation.cc:299] Negotiation successful
0812 01:23:37.764782 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":66,"threads_started":1}
I20260812 01:23:37.769305 30415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:37.766621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35982 (local address 127.24.0.188:32789)
0812 01:23:37.766739 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:37.766741 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:37.766752 (+ 11us) server_negotiation.cc:407] Connection header received
0812 01:23:37.766823 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:37.766824 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:37.766893 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:37.766992 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:23:37.767092 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:37.767579 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:37.768420 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:37.768622 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:37.768752 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:37.768933 (+ 181us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:23:37.769057 (+ 124us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:23:37.769165 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:37.769185 (+ 20us) server_negotiation.cc:299] Negotiation successful
0812 01:23:37.769205 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":62,"threads_started":1}
I20260812 01:23:37.774106 30415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:37.771293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35994 (local address 127.24.0.188:32789)
0812 01:23:37.771330 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:37.771332 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:37.771491 (+ 159us) server_negotiation.cc:407] Connection header received
0812 01:23:37.771519 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:37.771521 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:37.771532 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:37.771592 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:23:37.771748 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:37.772487 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:37.773337 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:37.773658 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:37.773731 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:37.773861 (+ 130us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:23:37.773893 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:23:37.773943 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:37.773994 (+ 51us) server_negotiation.cc:299] Negotiation successful
0812 01:23:37.774016 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 01:23:37.774550 30403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:37.771161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34640 (local address 127.24.0.189:38941)
0812 01:23:37.771222 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:37.771224 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:37.771317 (+ 93us) server_negotiation.cc:407] Connection header received
0812 01:23:37.771411 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:37.771412 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:37.771424 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:37.771482 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:23:37.771682 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:37.772444 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:37.773663 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:37.773958 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:37.774016 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:37.774195 (+ 179us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:23:37.774234 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:23:37.774264 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:37.774408 (+ 144us) server_negotiation.cc:299] Negotiation successful
0812 01:23:37.774433 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 01:23:37.774782 30428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:23:37.771382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46070 (local address 127.24.0.190:37259)
0812 01:23:37.771522 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0812 01:23:37.771524 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:23:37.771532 (+ 8us) server_negotiation.cc:407] Connection header received
0812 01:23:37.771580 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:23:37.771581 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:23:37.771637 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:23:37.771717 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:23:37.771874 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:37.772497 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:37.773634 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:23:37.774002 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:23:37.774240 (+ 238us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:23:37.774320 (+ 80us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:23:37.774434 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:23:37.774581 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0812 01:23:37.774660 (+ 79us) server_negotiation.cc:299] Negotiation successful
0812 01:23:37.774698 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":63,"threads_started":1}
I20260812 01:23:37.776633 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 29756
I20260812 01:23:37.798310 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 29866
I20260812 01:23:37.809234 24578 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoD5sF/build/release/bin/kudu with pid 29991
2026-08-12T01:23:37Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3167 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6268 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (283630 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST